[==========] Running 56 tests from 4 test suites.
[----------] Global test environment set-up.
[----------] 30 tests from SecurityITest
[ RUN ] SecurityITest.TestAuthorizationOnListTablets
Loading random data
Initializing database '/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786289732823074-3431-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 15:35:32 dist-test-slave-2w3w krb5kdc[3439](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 15:35:32 dist-test-slave-2w3w krb5kdc[3439](info): set up 2 sockets
Aug 09 15:35:32 dist-test-slave-2w3w krb5kdc[3439](info): commencing operation
krb5kdc: starting...
WARNING: Logging before InitGoogleLogging() is written to STDERR
W20260809 15:35:33.946239 3431 mini_kdc.cc:121] Time spent starting KDC: real 1.064s user 0.001s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786289732823074-3431-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786289732823074-3431-0/krb5kdc/test-user.keytab.
Aug 09 15:35:34 dist-test-slave-2w3w krb5kdc[3439](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786289734, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T15:35:34Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T15:35:34Z Disabled control of system clock
WARNING: no policy specified for kudu/127.3.89.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.3.89.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.3.89.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786289732823074-3431-0/krb5kdc/kudu_127.3.89.254.keytab.
Entry for principal kudu/127.3.89.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786289732823074-3431-0/krb5kdc/kudu_127.3.89.254.keytab.
I20260809 15:35:34.074653 3431 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskm314YR/build/debug/bin/kudu
/tmp/dist-test-taskm314YR/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786289732823074-3431-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786289732823074-3431-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786289732823074-3431-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786289732823074-3431-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.3.89.254:42915
--webserver_interface=127.3.89.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786289732823074-3431-0/krb5kdc/kudu_127.3.89.254.keytab
--principal=kudu/127.3.89.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.3.89.212:43419
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.3.89.254:42915
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786289732823074-3431-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786289732823074-3431-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786289732823074-3431-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 15:35:34.188700 3454 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 15:35:34.188977 3454 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 15:35:34.189018 3454 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 15:35:34.193121 3454 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 15:35:34.193177 3454 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 15:35:34.193197 3454 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 15:35:34.193213 3454 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 15:35:34.193269 3454 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 15:35:34.198596 3454 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.3.89.212:43419
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786289732823074-3431-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786289732823074-3431-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.3.89.254:42915
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786289732823074-3431-0/krb5kdc/kudu_127.3.89.254.keytab
--principal=kudu/127.3.89.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.3.89.254:42915
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786289732823074-3431-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.3.89.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.3454
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786289732823074-3431-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 09 Aug 2026 14:46:55 UTC on ccycloud.quasar-yswkmf.root.comops.site
build id 4576540
I20260809 15:35:34.200039 3454 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 15:35:34.201277 3454 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 15:35:34.213025 3463 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 15:35:34.213752 3461 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 15:35:34.214836 3454 server_base.cc:1034] running on GCE node
W20260809 15:35:34.215108 3460 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 15:35:34.215819 3454 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 15:35:34.217447 3454 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 15:35:34.218617 3454 hybrid_clock.cc:648] HybridClock initialized: now 1786289734218596 us; error 33 us; skew 500 ppm
Aug 09 15:35:34 dist-test-slave-2w3w krb5kdc[3439](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786289734, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.89.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 15:35:34.221822 3454 init.cc:377] Logged in from keytab as kudu/127.3.89.254@KRBTEST.COM (short username kudu)
I20260809 15:35:34.222888 3454 webserver.cc:460] Webserver started at http://127.3.89.254:42399/ using document root <none> and password file <none>
I20260809 15:35:34.223786 3454 fs_manager.cc:359] Metadata directory not provided
I20260809 15:35:34.223857 3454 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 15:35:34.224262 3454 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 15:35:34.227065 3454 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786289732823074-3431-0/minicluster-data/master-0/data/instance:
uuid: "e4fa8dc6a9e047f3924b9ca39babb8b6"
format_stamp: "Formatted at 2026-08-09 15:35:34 on dist-test-slave-2w3w"
server_key: "b2c093cdd1644b630604d5dd1f501e6a"
server_key_iv: "c4d4d6b324c21395b18423009ac561b4"
server_key_version: "encryptionkey@0"
I20260809 15:35:34.227661 3454 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786289732823074-3431-0/minicluster-data/master-0/wal/instance:
uuid: "e4fa8dc6a9e047f3924b9ca39babb8b6"
format_stamp: "Formatted at 2026-08-09 15:35:34 on dist-test-slave-2w3w"
server_key: "b2c093cdd1644b630604d5dd1f501e6a"
server_key_iv: "c4d4d6b324c21395b18423009ac561b4"
server_key_version: "encryptionkey@0"
I20260809 15:35:34.232488 3454 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.000s
I20260809 15:35:34.235808 3471 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 15:35:34.237298 3454 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260809 15:35:34.237610 3454 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786289732823074-3431-0/minicluster-data/master-0/data,/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786289732823074-3431-0/minicluster-data/master-0/wal
uuid: "e4fa8dc6a9e047f3924b9ca39babb8b6"
format_stamp: "Formatted at 2026-08-09 15:35:34 on dist-test-slave-2w3w"
server_key: "b2c093cdd1644b630604d5dd1f501e6a"
server_key_iv: "c4d4d6b324c21395b18423009ac561b4"
server_key_version: "encryptionkey@0"
I20260809 15:35:34.237731 3454 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786289732823074-3431-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786289732823074-3431-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786289732823074-3431-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 15:35:34.268374 3454 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 15:35:34.271966 3454 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 15:35:34.272202 3454 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 15:35:34.284237 3454 rpc_server.cc:307] RPC server started. Bound to: 127.3.89.254:42915
I20260809 15:35:34.284360 3524 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.3.89.254:42915 every 8 connection(s)
I20260809 15:35:34.285585 3454 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786289732823074-3431-0/minicluster-data/master-0/data/info.pb
I20260809 15:35:34.289953 3525 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 15:35:34.291980 3431 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskm314YR/build/debug/bin/kudu as pid 3454
I20260809 15:35:34.292158 3431 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786289732823074-3431-0/minicluster-data/master-0/wal/instance
I20260809 15:35:34.294139 3431 external_mini_cluster.cc:1428] Setting key 98eab9e7fb4e61492c2efff7357a3440
I20260809 15:35:34.299916 3525 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e4fa8dc6a9e047f3924b9ca39babb8b6: Bootstrap starting.
I20260809 15:35:34.303484 3525 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P e4fa8dc6a9e047f3924b9ca39babb8b6: Neither blocks nor log segments found. Creating new log.
I20260809 15:35:34.304886 3525 log.cc:824] T 00000000000000000000000000000000 P e4fa8dc6a9e047f3924b9ca39babb8b6: Log is configured to *not* fsync() on all Append() calls
I20260809 15:35:34.308070 3525 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e4fa8dc6a9e047f3924b9ca39babb8b6: No bootstrap required, opened a new log
I20260809 15:35:34.314427 3525 raft_consensus.cc:348] T 00000000000000000000000000000000 P e4fa8dc6a9e047f3924b9ca39babb8b6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e4fa8dc6a9e047f3924b9ca39babb8b6" member_type: VOTER last_known_addr { host: "127.3.89.254" port: 42915 } }
I20260809 15:35:34.314754 3525 raft_consensus.cc:374] T 00000000000000000000000000000000 P e4fa8dc6a9e047f3924b9ca39babb8b6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 15:35:34.314862 3525 raft_consensus.cc:729] T 00000000000000000000000000000000 P e4fa8dc6a9e047f3924b9ca39babb8b6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e4fa8dc6a9e047f3924b9ca39babb8b6, State: Initialized, Role: FOLLOWER
I20260809 15:35:34.315610 3525 consensus_queue.cc:260] T 00000000000000000000000000000000 P e4fa8dc6a9e047f3924b9ca39babb8b6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e4fa8dc6a9e047f3924b9ca39babb8b6" member_type: VOTER last_known_addr { host: "127.3.89.254" port: 42915 } }
I20260809 15:35:34.315832 3525 raft_consensus.cc:388] T 00000000000000000000000000000000 P e4fa8dc6a9e047f3924b9ca39babb8b6 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 15:35:34.315972 3525 raft_consensus.cc:482] T 00000000000000000000000000000000 P e4fa8dc6a9e047f3924b9ca39babb8b6 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 15:35:34.316130 3525 raft_consensus.cc:3037] T 00000000000000000000000000000000 P e4fa8dc6a9e047f3924b9ca39babb8b6 [term 0 FOLLOWER]: Advancing to term 1
I20260809 15:35:34.317139 3525 raft_consensus.cc:504] T 00000000000000000000000000000000 P e4fa8dc6a9e047f3924b9ca39babb8b6 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e4fa8dc6a9e047f3924b9ca39babb8b6" member_type: VOTER last_known_addr { host: "127.3.89.254" port: 42915 } }
I20260809 15:35:34.317791 3525 leader_election.cc:302] T 00000000000000000000000000000000 P e4fa8dc6a9e047f3924b9ca39babb8b6 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: e4fa8dc6a9e047f3924b9ca39babb8b6; no voters:
Aug 09 15:35:34 dist-test-slave-2w3w krb5kdc[3439](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786289734, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.3.89.254@KRBTEST.COM
I20260809 15:35:34.318228 3525 leader_election.cc:288] T 00000000000000000000000000000000 P e4fa8dc6a9e047f3924b9ca39babb8b6 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 15:35:34.318435 3531 raft_consensus.cc:2781] T 00000000000000000000000000000000 P e4fa8dc6a9e047f3924b9ca39babb8b6 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 15:35:34.318924 3531 raft_consensus.cc:686] T 00000000000000000000000000000000 P e4fa8dc6a9e047f3924b9ca39babb8b6 [term 1 LEADER]: Becoming Leader. State: Replica: e4fa8dc6a9e047f3924b9ca39babb8b6, State: Running, Role: LEADER
I20260809 15:35:34.319591 3531 consensus_queue.cc:237] T 00000000000000000000000000000000 P e4fa8dc6a9e047f3924b9ca39babb8b6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e4fa8dc6a9e047f3924b9ca39babb8b6" member_type: VOTER last_known_addr { host: "127.3.89.254" port: 42915 } }
I20260809 15:35:34.320171 3525 sys_catalog.cc:564] T 00000000000000000000000000000000 P e4fa8dc6a9e047f3924b9ca39babb8b6 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 15:35:34.322626 3533 sys_catalog.cc:455] T 00000000000000000000000000000000 P e4fa8dc6a9e047f3924b9ca39babb8b6 [sys.catalog]: SysCatalogTable state changed. Reason: New leader e4fa8dc6a9e047f3924b9ca39babb8b6. Latest consensus state: current_term: 1 leader_uuid: "e4fa8dc6a9e047f3924b9ca39babb8b6" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e4fa8dc6a9e047f3924b9ca39babb8b6" member_type: VOTER last_known_addr { host: "127.3.89.254" port: 42915 } } }
I20260809 15:35:34.322768 3533 sys_catalog.cc:458] T 00000000000000000000000000000000 P e4fa8dc6a9e047f3924b9ca39babb8b6 [sys.catalog]: This master's current role is: LEADER
I20260809 15:35:34.323199 3532 sys_catalog.cc:455] T 00000000000000000000000000000000 P e4fa8dc6a9e047f3924b9ca39babb8b6 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "e4fa8dc6a9e047f3924b9ca39babb8b6" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e4fa8dc6a9e047f3924b9ca39babb8b6" member_type: VOTER last_known_addr { host: "127.3.89.254" port: 42915 } } }
I20260809 15:35:34.323340 3532 sys_catalog.cc:458] T 00000000000000000000000000000000 P e4fa8dc6a9e047f3924b9ca39babb8b6 [sys.catalog]: This master's current role is: LEADER
I20260809 15:35:34.328601 3527 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:35:34.299660 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37024 (local address 127.3.89.254:42915)
0809 15:35:34.300356 (+ 696us) server_negotiation.cc:206] Beginning negotiation
0809 15:35:34.300420 (+ 64us) server_negotiation.cc:399] Waiting for connection header
0809 15:35:34.301600 (+ 1180us) server_negotiation.cc:407] Connection header received
0809 15:35:34.306581 (+ 4981us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 15:35:34.306601 (+ 20us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 15:35:34.307131 (+ 530us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 15:35:34.307534 (+ 403us) server_negotiation.cc:226] Negotiated authn=SASL
0809 15:35:34.309943 (+ 2409us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:35:34.311858 (+ 1915us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:35:34.313142 (+ 1284us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:35:34.313936 (+ 794us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:35:34.314183 (+ 247us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:35:34.319437 (+ 5254us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 15:35:34.319480 (+ 43us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 15:35:34.319505 (+ 25us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 15:35:34.319537 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0809 15:35:34.322802 (+ 3265us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 15:35:34.323756 (+ 954us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 15:35:34.323764 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 15:35:34.323770 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0809 15:35:34.323935 (+ 165us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 15:35:34.324486 (+ 551us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 15:35:34.324490 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 15:35:34.324492 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 15:35:34.325338 (+ 846us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 15:35:34.325669 (+ 331us) server_negotiation.cc:1035] Waiting for connection context
0809 15:35:34.326238 (+ 569us) server_negotiation.cc:299] Negotiation successful
0809 15:35:34.327120 (+ 882us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":522,"thread_start_us":206,"threads_started":1}
W20260809 15:35:34.330438 3548 catalog_manager.cc:1558] T 00000000000000000000000000000000 P e4fa8dc6a9e047f3924b9ca39babb8b6: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260809 15:35:34.330571 3548 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260809 15:35:34.330693 3547 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 15:35:34.331820 3547 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 15:35:34.340273 3547 catalog_manager.cc:1347] Generated new cluster ID: 2906b84ede214f59b09670210c58bbb7
I20260809 15:35:34.340354 3547 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 15:35:34.350221 3547 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 15:35:34.351351 3547 catalog_manager.cc:1504] Loading token signing keys...
I20260809 15:35:34.363091 3547 catalog_manager.cc:5868] T 00000000000000000000000000000000 P e4fa8dc6a9e047f3924b9ca39babb8b6: Generated new TSK 0
I20260809 15:35:34.364212 3547 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.3.89.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.3.89.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.3.89.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786289732823074-3431-0/krb5kdc/kudu_127.3.89.193.keytab.
Entry for principal kudu/127.3.89.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786289732823074-3431-0/krb5kdc/kudu_127.3.89.193.keytab.
I20260809 15:35:34.400305 3431 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskm314YR/build/debug/bin/kudu
/tmp/dist-test-taskm314YR/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786289732823074-3431-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786289732823074-3431-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786289732823074-3431-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786289732823074-3431-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.3.89.193:0
--local_ip_for_outbound_sockets=127.3.89.193
--webserver_interface=127.3.89.193
--webserver_port=0
--tserver_master_addrs=127.3.89.254:42915
--keytab_file=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786289732823074-3431-0/krb5kdc/kudu_127.3.89.193.keytab
--principal=kudu/127.3.89.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.3.89.212:43419
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786289732823074-3431-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786289732823074-3431-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786289732823074-3431-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 15:35:34.534857 3554 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 15:35:34.535122 3554 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 15:35:34.535197 3554 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 15:35:34.539265 3554 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 15:35:34.539330 3554 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 15:35:34.539433 3554 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.3.89.193
I20260809 15:35:34.544951 3554 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.3.89.212:43419
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786289732823074-3431-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786289732823074-3431-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786289732823074-3431-0/krb5kdc/kudu_127.3.89.193.keytab
--principal=kudu/127.3.89.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.3.89.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786289732823074-3431-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.3.89.193
--webserver_port=0
--tserver_master_addrs=127.3.89.254:42915
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.3554
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.3.89.193
--log_dir=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786289732823074-3431-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 09 Aug 2026 14:46:55 UTC on ccycloud.quasar-yswkmf.root.comops.site
build id 4576540
I20260809 15:35:34.545989 3554 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 15:35:34.547008 3554 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 15:35:34.558049 3563 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 15:35:34.558041 3560 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 15:35:34.558260 3554 server_base.cc:1034] running on GCE node
W20260809 15:35:34.558041 3561 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 15:35:34.558935 3554 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 15:35:34.559664 3554 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 15:35:34.560882 3554 hybrid_clock.cc:648] HybridClock initialized: now 1786289734560871 us; error 28 us; skew 500 ppm
Aug 09 15:35:34 dist-test-slave-2w3w krb5kdc[3439](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786289734, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.89.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 15:35:34.564230 3554 init.cc:377] Logged in from keytab as kudu/127.3.89.193@KRBTEST.COM (short username kudu)
I20260809 15:35:34.565073 3554 webserver.cc:460] Webserver started at http://127.3.89.193:38971/ using document root <none> and password file <none>
I20260809 15:35:34.565623 3554 fs_manager.cc:359] Metadata directory not provided
I20260809 15:35:34.565708 3554 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 15:35:34.565948 3554 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 15:35:34.568142 3554 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786289732823074-3431-0/minicluster-data/ts-0/data/instance:
uuid: "98aa27443a5d461db5b52f8d8710f02a"
format_stamp: "Formatted at 2026-08-09 15:35:34 on dist-test-slave-2w3w"
server_key: "114c22a11aa4371cb3563759b912c8e1"
server_key_iv: "30bcd7f6076e39030ffa4e20b8f97fe0"
server_key_version: "encryptionkey@0"
I20260809 15:35:34.568699 3554 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786289732823074-3431-0/minicluster-data/ts-0/wal/instance:
uuid: "98aa27443a5d461db5b52f8d8710f02a"
format_stamp: "Formatted at 2026-08-09 15:35:34 on dist-test-slave-2w3w"
server_key: "114c22a11aa4371cb3563759b912c8e1"
server_key_iv: "30bcd7f6076e39030ffa4e20b8f97fe0"
server_key_version: "encryptionkey@0"
I20260809 15:35:34.572685 3554 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260809 15:35:34.575109 3570 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 15:35:34.576229 3554 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.001s
I20260809 15:35:34.576363 3554 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786289732823074-3431-0/minicluster-data/ts-0/data,/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786289732823074-3431-0/minicluster-data/ts-0/wal
uuid: "98aa27443a5d461db5b52f8d8710f02a"
format_stamp: "Formatted at 2026-08-09 15:35:34 on dist-test-slave-2w3w"
server_key: "114c22a11aa4371cb3563759b912c8e1"
server_key_iv: "30bcd7f6076e39030ffa4e20b8f97fe0"
server_key_version: "encryptionkey@0"
I20260809 15:35:34.576465 3554 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786289732823074-3431-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786289732823074-3431-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786289732823074-3431-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 15:35:34.600047 3554 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 15:35:34.603183 3554 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 15:35:34.603433 3554 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 15:35:34.604300 3554 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 15:35:34.605505 3554 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 15:35:34.605580 3554 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 15:35:34.605651 3554 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 15:35:34.605691 3554 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 15:35:34.619747 3554 rpc_server.cc:307] RPC server started. Bound to: 127.3.89.193:34645
I20260809 15:35:34.619879 3694 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.3.89.193:34645 every 8 connection(s)
I20260809 15:35:34.620769 3554 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786289732823074-3431-0/minicluster-data/ts-0/data/info.pb
I20260809 15:35:34.627444 3431 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskm314YR/build/debug/bin/kudu as pid 3554
I20260809 15:35:34.627555 3431 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786289732823074-3431-0/minicluster-data/ts-0/wal/instance
I20260809 15:35:34.627851 3431 external_mini_cluster.cc:1428] Setting key 3b66088b308e1d36997c1d739338e2cb
Aug 09 15:35:34 dist-test-slave-2w3w krb5kdc[3439](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786289734, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.89.193@KRBTEST.COM for kudu/127.3.89.254@KRBTEST.COM
I20260809 15:35:34.638198 3527 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:35:34.623095 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.89.193:43991 (local address 127.3.89.254:42915)
0809 15:35:34.623228 (+ 133us) server_negotiation.cc:206] Beginning negotiation
0809 15:35:34.623232 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 15:35:34.624197 (+ 965us) server_negotiation.cc:407] Connection header received
0809 15:35:34.626123 (+ 1926us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 15:35:34.626126 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 15:35:34.626187 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 15:35:34.626266 (+ 79us) server_negotiation.cc:226] Negotiated authn=SASL
0809 15:35:34.627925 (+ 1659us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:35:34.628682 (+ 757us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:35:34.630471 (+ 1789us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:35:34.630702 (+ 231us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:35:34.630794 (+ 92us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:35:34.634254 (+ 3460us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 15:35:34.634257 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 15:35:34.634259 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 15:35:34.634290 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0809 15:35:34.635913 (+ 1623us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 15:35:34.636594 (+ 681us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 15:35:34.636596 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 15:35:34.636597 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 15:35:34.636674 (+ 77us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 15:35:34.637100 (+ 426us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 15:35:34.637102 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 15:35:34.637103 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 15:35:34.637267 (+ 164us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 15:35:34.637340 (+ 73us) server_negotiation.cc:1035] Waiting for connection context
0809 15:35:34.637966 (+ 626us) server_negotiation.cc:299] Negotiation successful
0809 15:35:34.638079 (+ 113us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":46}
I20260809 15:35:34.639447 3697 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:35:34.623497 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.89.254:42915 (local address 127.3.89.193:43991)
0809 15:35:34.624052 (+ 555us) negotiation.cc:107] Waiting for socket to connect
0809 15:35:34.624094 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0809 15:35:34.625895 (+ 1801us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 15:35:34.626442 (+ 547us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 15:35:34.626449 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 15:35:34.627001 (+ 552us) client_negotiation.cc:189] Negotiated authn=SASL
0809 15:35:34.627735 (+ 734us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:35:34.627755 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:35:34.628829 (+ 1074us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:35:34.628832 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:35:34.630334 (+ 1502us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:35:34.630350 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:35:34.630869 (+ 519us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:35:34.630875 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:35:34.631200 (+ 325us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:35:34.632084 (+ 884us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 15:35:34.632099 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0809 15:35:34.634087 (+ 1988us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 15:35:34.636038 (+ 1951us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 15:35:34.636047 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 15:35:34.636065 (+ 18us) client_negotiation.cc:760] calling sasl_client_step()
0809 15:35:34.636478 (+ 413us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 15:35:34.636763 (+ 285us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 15:35:34.636768 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 15:35:34.636773 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0809 15:35:34.636998 (+ 225us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 15:35:34.637371 (+ 373us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 15:35:34.637377 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 15:35:34.637696 (+ 319us) client_negotiation.cc:769] Sending connection context
0809 15:35:34.637925 (+ 229us) client_negotiation.cc:240] Negotiation successful
0809 15:35:34.638275 (+ 350us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":383,"thread_start_us":199,"threads_started":1}
WARNING: no policy specified for kudu/127.3.89.194@KRBTEST.COM; defaulting to no policy
I20260809 15:35:34.640967 3695 heartbeater.cc:347] Connected to a master server at 127.3.89.254:42915
I20260809 15:35:34.641170 3695 heartbeater.cc:464] Registering TS with master...
I20260809 15:35:34.641669 3695 heartbeater.cc:511] Master 127.3.89.254:42915 requested a full tablet report, sending...
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.3.89.194@KRBTEST.COM" created.
I20260809 15:35:34.643787 3488 ts_manager.cc:194] Registered new tserver with Master: 98aa27443a5d461db5b52f8d8710f02a (127.3.89.193:34645)
I20260809 15:35:34.645808 3488 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.3.89.193@KRBTEST.COM'} at 127.3.89.193:43991
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.3.89.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786289732823074-3431-0/krb5kdc/kudu_127.3.89.194.keytab.
Entry for principal kudu/127.3.89.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786289732823074-3431-0/krb5kdc/kudu_127.3.89.194.keytab.
I20260809 15:35:34.657642 3431 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskm314YR/build/debug/bin/kudu
/tmp/dist-test-taskm314YR/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786289732823074-3431-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786289732823074-3431-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786289732823074-3431-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786289732823074-3431-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.3.89.194:0
--local_ip_for_outbound_sockets=127.3.89.194
--webserver_interface=127.3.89.194
--webserver_port=0
--tserver_master_addrs=127.3.89.254:42915
--keytab_file=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786289732823074-3431-0/krb5kdc/kudu_127.3.89.194.keytab
--principal=kudu/127.3.89.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.3.89.212:43419
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786289732823074-3431-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786289732823074-3431-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786289732823074-3431-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 15:35:34.766134 3701 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 15:35:34.766381 3701 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 15:35:34.766424 3701 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 15:35:34.770514 3701 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 15:35:34.770572 3701 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 15:35:34.770628 3701 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.3.89.194
I20260809 15:35:34.775795 3701 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.3.89.212:43419
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786289732823074-3431-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786289732823074-3431-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786289732823074-3431-0/krb5kdc/kudu_127.3.89.194.keytab
--principal=kudu/127.3.89.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.3.89.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786289732823074-3431-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.3.89.194
--webserver_port=0
--tserver_master_addrs=127.3.89.254:42915
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.3701
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.3.89.194
--log_dir=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786289732823074-3431-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 09 Aug 2026 14:46:55 UTC on ccycloud.quasar-yswkmf.root.comops.site
build id 4576540
I20260809 15:35:34.776819 3701 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 15:35:34.777817 3701 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 15:35:34.787140 3707 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 15:35:34.787338 3701 server_base.cc:1034] running on GCE node
W20260809 15:35:34.787146 3708 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 15:35:34.787146 3710 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 15:35:34.787868 3701 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 15:35:34.788470 3701 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 15:35:34.789649 3701 hybrid_clock.cc:648] HybridClock initialized: now 1786289734789634 us; error 27 us; skew 500 ppm
Aug 09 15:35:34 dist-test-slave-2w3w krb5kdc[3439](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786289734, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.89.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 15:35:34.792390 3701 init.cc:377] Logged in from keytab as kudu/127.3.89.194@KRBTEST.COM (short username kudu)
I20260809 15:35:34.793251 3701 webserver.cc:460] Webserver started at http://127.3.89.194:37767/ using document root <none> and password file <none>
I20260809 15:35:34.793740 3701 fs_manager.cc:359] Metadata directory not provided
I20260809 15:35:34.793798 3701 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 15:35:34.793954 3701 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 15:35:34.795912 3701 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786289732823074-3431-0/minicluster-data/ts-1/data/instance:
uuid: "ee6d268477fc4596a5ab06d9b156ff38"
format_stamp: "Formatted at 2026-08-09 15:35:34 on dist-test-slave-2w3w"
server_key: "240535a8e6ec50d4a77b4eaf9ed1b016"
server_key_iv: "981b8fe737e2784ca4c6a1a10d1de9eb"
server_key_version: "encryptionkey@0"
I20260809 15:35:34.796466 3701 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786289732823074-3431-0/minicluster-data/ts-1/wal/instance:
uuid: "ee6d268477fc4596a5ab06d9b156ff38"
format_stamp: "Formatted at 2026-08-09 15:35:34 on dist-test-slave-2w3w"
server_key: "240535a8e6ec50d4a77b4eaf9ed1b016"
server_key_iv: "981b8fe737e2784ca4c6a1a10d1de9eb"
server_key_version: "encryptionkey@0"
I20260809 15:35:34.799928 3701 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.003s sys 0.001s
I20260809 15:35:34.802088 3718 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 15:35:34.803290 3701 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260809 15:35:34.803386 3701 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786289732823074-3431-0/minicluster-data/ts-1/data,/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786289732823074-3431-0/minicluster-data/ts-1/wal
uuid: "ee6d268477fc4596a5ab06d9b156ff38"
format_stamp: "Formatted at 2026-08-09 15:35:34 on dist-test-slave-2w3w"
server_key: "240535a8e6ec50d4a77b4eaf9ed1b016"
server_key_iv: "981b8fe737e2784ca4c6a1a10d1de9eb"
server_key_version: "encryptionkey@0"
I20260809 15:35:34.803465 3701 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786289732823074-3431-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786289732823074-3431-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786289732823074-3431-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 15:35:34.830802 3701 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 15:35:34.834012 3701 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 15:35:34.834163 3701 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 15:35:34.834658 3701 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 15:35:34.835469 3701 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 15:35:34.835515 3701 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 15:35:34.835554 3701 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 15:35:34.835574 3701 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 15:35:34.844360 3701 rpc_server.cc:307] RPC server started. Bound to: 127.3.89.194:36321
I20260809 15:35:34.844409 3842 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.3.89.194:36321 every 8 connection(s)
I20260809 15:35:34.845213 3701 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786289732823074-3431-0/minicluster-data/ts-1/data/info.pb
I20260809 15:35:34.853912 3431 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskm314YR/build/debug/bin/kudu as pid 3701
I20260809 15:35:34.854051 3431 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786289732823074-3431-0/minicluster-data/ts-1/wal/instance
I20260809 15:35:34.854344 3431 external_mini_cluster.cc:1428] Setting key 0e2f1f82ccc67afe8d516485b4fb9a3c
Aug 09 15:35:34 dist-test-slave-2w3w krb5kdc[3439](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786289734, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.89.194@KRBTEST.COM for kudu/127.3.89.254@KRBTEST.COM
I20260809 15:35:34.858731 3527 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:35:34.846886 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.89.194:46141 (local address 127.3.89.254:42915)
0809 15:35:34.847063 (+ 177us) server_negotiation.cc:206] Beginning negotiation
0809 15:35:34.847067 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 15:35:34.847893 (+ 826us) server_negotiation.cc:407] Connection header received
0809 15:35:34.849231 (+ 1338us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 15:35:34.849234 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 15:35:34.849295 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 15:35:34.849398 (+ 103us) server_negotiation.cc:226] Negotiated authn=SASL
0809 15:35:34.850527 (+ 1129us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:35:34.851147 (+ 620us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:35:34.852285 (+ 1138us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:35:34.852512 (+ 227us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:35:34.852618 (+ 106us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:35:34.855218 (+ 2600us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 15:35:34.855221 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 15:35:34.855223 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 15:35:34.855247 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0809 15:35:34.856730 (+ 1483us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 15:35:34.857238 (+ 508us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 15:35:34.857242 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 15:35:34.857244 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 15:35:34.857326 (+ 82us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 15:35:34.857666 (+ 340us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 15:35:34.857670 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 15:35:34.857671 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 15:35:34.857853 (+ 182us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 15:35:34.857968 (+ 115us) server_negotiation.cc:1035] Waiting for connection context
0809 15:35:34.858441 (+ 473us) server_negotiation.cc:299] Negotiation successful
0809 15:35:34.858562 (+ 121us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":50}
I20260809 15:35:34.859561 3846 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:35:34.847276 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.89.254:42915 (local address 127.3.89.194:46141)
0809 15:35:34.847737 (+ 461us) negotiation.cc:107] Waiting for socket to connect
0809 15:35:34.847780 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0809 15:35:34.849027 (+ 1247us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 15:35:34.849566 (+ 539us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 15:35:34.849572 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 15:35:34.849937 (+ 365us) client_negotiation.cc:189] Negotiated authn=SASL
0809 15:35:34.850389 (+ 452us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:35:34.850402 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:35:34.851264 (+ 862us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:35:34.851268 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:35:34.852158 (+ 890us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:35:34.852167 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:35:34.852656 (+ 489us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:35:34.852660 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:35:34.852824 (+ 164us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:35:34.853354 (+ 530us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 15:35:34.853370 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0809 15:35:34.855063 (+ 1693us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 15:35:34.856852 (+ 1789us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 15:35:34.856858 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 15:35:34.856873 (+ 15us) client_negotiation.cc:760] calling sasl_client_step()
0809 15:35:34.857133 (+ 260us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 15:35:34.857423 (+ 290us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 15:35:34.857426 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 15:35:34.857428 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 15:35:34.857568 (+ 140us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 15:35:34.857958 (+ 390us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 15:35:34.857965 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 15:35:34.858200 (+ 235us) client_negotiation.cc:769] Sending connection context
0809 15:35:34.858378 (+ 178us) client_negotiation.cc:240] Negotiation successful
0809 15:35:34.858698 (+ 320us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":289,"thread_start_us":122,"threads_started":1}
I20260809 15:35:34.860672 3844 heartbeater.cc:347] Connected to a master server at 127.3.89.254:42915
I20260809 15:35:34.860929 3844 heartbeater.cc:464] Registering TS with master...
I20260809 15:35:34.861511 3844 heartbeater.cc:511] Master 127.3.89.254:42915 requested a full tablet report, sending...
I20260809 15:35:34.862762 3488 ts_manager.cc:194] Registered new tserver with Master: ee6d268477fc4596a5ab06d9b156ff38 (127.3.89.194:36321)
I20260809 15:35:34.863673 3488 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.3.89.194@KRBTEST.COM'} at 127.3.89.194:46141
WARNING: no policy specified for kudu/127.3.89.195@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.3.89.195@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.3.89.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786289732823074-3431-0/krb5kdc/kudu_127.3.89.195.keytab.
Entry for principal kudu/127.3.89.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786289732823074-3431-0/krb5kdc/kudu_127.3.89.195.keytab.
I20260809 15:35:34.882541 3431 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskm314YR/build/debug/bin/kudu
/tmp/dist-test-taskm314YR/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786289732823074-3431-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786289732823074-3431-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786289732823074-3431-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786289732823074-3431-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.3.89.195:0
--local_ip_for_outbound_sockets=127.3.89.195
--webserver_interface=127.3.89.195
--webserver_port=0
--tserver_master_addrs=127.3.89.254:42915
--keytab_file=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786289732823074-3431-0/krb5kdc/kudu_127.3.89.195.keytab
--principal=kudu/127.3.89.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.3.89.212:43419
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786289732823074-3431-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786289732823074-3431-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786289732823074-3431-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 15:35:34.994298 3849 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 15:35:34.994537 3849 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 15:35:34.994619 3849 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 15:35:34.998641 3849 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 15:35:34.998696 3849 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 15:35:34.998781 3849 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.3.89.195
I20260809 15:35:35.003895 3849 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.3.89.212:43419
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786289732823074-3431-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786289732823074-3431-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786289732823074-3431-0/krb5kdc/kudu_127.3.89.195.keytab
--principal=kudu/127.3.89.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.3.89.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786289732823074-3431-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.3.89.195
--webserver_port=0
--tserver_master_addrs=127.3.89.254:42915
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.3849
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.3.89.195
--log_dir=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786289732823074-3431-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 09 Aug 2026 14:46:55 UTC on ccycloud.quasar-yswkmf.root.comops.site
build id 4576540
I20260809 15:35:35.004889 3849 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 15:35:35.005862 3849 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260809 15:35:35.015498 3849 server_base.cc:1034] running on GCE node
W20260809 15:35:35.015537 3856 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 15:35:35.015547 3854 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 15:35:35.015705 3858 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 15:35:35.016067 3849 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 15:35:35.016726 3849 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 15:35:35.017897 3849 hybrid_clock.cc:648] HybridClock initialized: now 1786289735017882 us; error 26 us; skew 500 ppm
Aug 09 15:35:35 dist-test-slave-2w3w krb5kdc[3439](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786289735, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.89.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 15:35:35.021065 3849 init.cc:377] Logged in from keytab as kudu/127.3.89.195@KRBTEST.COM (short username kudu)
I20260809 15:35:35.021997 3849 webserver.cc:460] Webserver started at http://127.3.89.195:33363/ using document root <none> and password file <none>
I20260809 15:35:35.022543 3849 fs_manager.cc:359] Metadata directory not provided
I20260809 15:35:35.022632 3849 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 15:35:35.022858 3849 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 15:35:35.024853 3849 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786289732823074-3431-0/minicluster-data/ts-2/data/instance:
uuid: "dafb6c0ac9084130a63ba450e8b056aa"
format_stamp: "Formatted at 2026-08-09 15:35:35 on dist-test-slave-2w3w"
server_key: "98628b0fda807bbbab009cd42700a18b"
server_key_iv: "5569e15744e0cbfb877a67bed5d921c6"
server_key_version: "encryptionkey@0"
I20260809 15:35:35.025401 3849 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786289732823074-3431-0/minicluster-data/ts-2/wal/instance:
uuid: "dafb6c0ac9084130a63ba450e8b056aa"
format_stamp: "Formatted at 2026-08-09 15:35:35 on dist-test-slave-2w3w"
server_key: "98628b0fda807bbbab009cd42700a18b"
server_key_iv: "5569e15744e0cbfb877a67bed5d921c6"
server_key_version: "encryptionkey@0"
I20260809 15:35:35.029397 3849 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260809 15:35:35.031698 3865 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 15:35:35.032866 3849 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260809 15:35:35.032992 3849 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786289732823074-3431-0/minicluster-data/ts-2/data,/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786289732823074-3431-0/minicluster-data/ts-2/wal
uuid: "dafb6c0ac9084130a63ba450e8b056aa"
format_stamp: "Formatted at 2026-08-09 15:35:35 on dist-test-slave-2w3w"
server_key: "98628b0fda807bbbab009cd42700a18b"
server_key_iv: "5569e15744e0cbfb877a67bed5d921c6"
server_key_version: "encryptionkey@0"
I20260809 15:35:35.033095 3849 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786289732823074-3431-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786289732823074-3431-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786289732823074-3431-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 15:35:35.058934 3849 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 15:35:35.061784 3849 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 15:35:35.061986 3849 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 15:35:35.062580 3849 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 15:35:35.063453 3849 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 15:35:35.063522 3849 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 15:35:35.063591 3849 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 15:35:35.063637 3849 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 15:35:35.072818 3849 rpc_server.cc:307] RPC server started. Bound to: 127.3.89.195:43953
I20260809 15:35:35.072849 3987 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.3.89.195:43953 every 8 connection(s)
I20260809 15:35:35.073673 3849 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786289732823074-3431-0/minicluster-data/ts-2/data/info.pb
I20260809 15:35:35.079416 3431 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskm314YR/build/debug/bin/kudu as pid 3849
I20260809 15:35:35.079540 3431 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786289732823074-3431-0/minicluster-data/ts-2/wal/instance
I20260809 15:35:35.079808 3431 external_mini_cluster.cc:1428] Setting key b248a125f0aa5191812ab6fe0d2a8ba1
Aug 09 15:35:35 dist-test-slave-2w3w krb5kdc[3439](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786289735, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.89.195@KRBTEST.COM for kudu/127.3.89.254@KRBTEST.COM
I20260809 15:35:35.087939 3527 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:35:35.075584 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.89.195:49581 (local address 127.3.89.254:42915)
0809 15:35:35.075724 (+ 140us) server_negotiation.cc:206] Beginning negotiation
0809 15:35:35.075729 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 15:35:35.076567 (+ 838us) server_negotiation.cc:407] Connection header received
0809 15:35:35.077851 (+ 1284us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 15:35:35.077853 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 15:35:35.077900 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 15:35:35.077974 (+ 74us) server_negotiation.cc:226] Negotiated authn=SASL
0809 15:35:35.079103 (+ 1129us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:35:35.079979 (+ 876us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:35:35.081448 (+ 1469us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:35:35.081846 (+ 398us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:35:35.081975 (+ 129us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:35:35.084456 (+ 2481us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 15:35:35.084459 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 15:35:35.084461 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 15:35:35.084486 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0809 15:35:35.085966 (+ 1480us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 15:35:35.086401 (+ 435us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 15:35:35.086403 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 15:35:35.086404 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 15:35:35.086480 (+ 76us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 15:35:35.086887 (+ 407us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 15:35:35.086889 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 15:35:35.086890 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 15:35:35.087114 (+ 224us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 15:35:35.087194 (+ 80us) server_negotiation.cc:1035] Waiting for connection context
0809 15:35:35.087704 (+ 510us) server_negotiation.cc:299] Negotiation successful
0809 15:35:35.087823 (+ 119us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":49}
I20260809 15:35:35.088757 3990 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:35:35.075983 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.89.254:42915 (local address 127.3.89.195:49581)
0809 15:35:35.076421 (+ 438us) negotiation.cc:107] Waiting for socket to connect
0809 15:35:35.076464 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0809 15:35:35.077638 (+ 1174us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 15:35:35.078124 (+ 486us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 15:35:35.078130 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 15:35:35.078470 (+ 340us) client_negotiation.cc:189] Negotiated authn=SASL
0809 15:35:35.078937 (+ 467us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:35:35.078949 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:35:35.080101 (+ 1152us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:35:35.080104 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:35:35.081323 (+ 1219us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:35:35.081334 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:35:35.082009 (+ 675us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:35:35.082011 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:35:35.082152 (+ 141us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:35:35.082799 (+ 647us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 15:35:35.082818 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0809 15:35:35.084308 (+ 1490us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 15:35:35.086061 (+ 1753us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 15:35:35.086065 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 15:35:35.086074 (+ 9us) client_negotiation.cc:760] calling sasl_client_step()
0809 15:35:35.086313 (+ 239us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 15:35:35.086591 (+ 278us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 15:35:35.086594 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 15:35:35.086596 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 15:35:35.086753 (+ 157us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 15:35:35.087211 (+ 458us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 15:35:35.087216 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 15:35:35.087447 (+ 231us) client_negotiation.cc:769] Sending connection context
0809 15:35:35.087622 (+ 175us) client_negotiation.cc:240] Negotiation successful
0809 15:35:35.087915 (+ 293us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":266,"thread_start_us":129,"threads_started":1}
I20260809 15:35:35.089910 3988 heartbeater.cc:347] Connected to a master server at 127.3.89.254:42915
I20260809 15:35:35.090143 3988 heartbeater.cc:464] Registering TS with master...
I20260809 15:35:35.090622 3988 heartbeater.cc:511] Master 127.3.89.254:42915 requested a full tablet report, sending...
I20260809 15:35:35.091653 3488 ts_manager.cc:194] Registered new tserver with Master: dafb6c0ac9084130a63ba450e8b056aa (127.3.89.195:43953)
I20260809 15:35:35.092206 3488 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.3.89.195@KRBTEST.COM'} at 127.3.89.195:49581
I20260809 15:35:35.094497 3431 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Aug 09 15:35:35 dist-test-slave-2w3w krb5kdc[3439](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786289735, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Aug 09 15:35:35 dist-test-slave-2w3w krb5kdc[3439](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786289735, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.3.89.193@KRBTEST.COM
I20260809 15:35:35.121250 3996 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:35:35.111980 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33140 (local address 127.3.89.193:34645)
0809 15:35:35.112317 (+ 337us) server_negotiation.cc:206] Beginning negotiation
0809 15:35:35.112333 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0809 15:35:35.112357 (+ 24us) server_negotiation.cc:407] Connection header received
0809 15:35:35.112409 (+ 52us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 15:35:35.112414 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 15:35:35.112604 (+ 190us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 15:35:35.112735 (+ 131us) server_negotiation.cc:226] Negotiated authn=SASL
0809 15:35:35.113363 (+ 628us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:35:35.114217 (+ 854us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:35:35.115112 (+ 895us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:35:35.115484 (+ 372us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:35:35.115579 (+ 95us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:35:35.117834 (+ 2255us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 15:35:35.117843 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 15:35:35.117855 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 15:35:35.117876 (+ 21us) server_negotiation.cc:922] calling sasl_server_start()
0809 15:35:35.119453 (+ 1577us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 15:35:35.119843 (+ 390us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 15:35:35.119848 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 15:35:35.119849 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 15:35:35.119930 (+ 81us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 15:35:35.120204 (+ 274us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 15:35:35.120207 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 15:35:35.120208 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 15:35:35.120513 (+ 305us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 15:35:35.120692 (+ 179us) server_negotiation.cc:1035] Waiting for connection context
0809 15:35:35.120900 (+ 208us) server_negotiation.cc:299] Negotiation successful
0809 15:35:35.121062 (+ 162us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":215,"thread_start_us":89,"threads_started":1}
W20260809 15:35:35.122383 3601 server_base.cc:1116] Unauthorized access attempt to method kudu.tserver.TabletServerService.ListTablets from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:33140
Aug 09 15:35:35 dist-test-slave-2w3w krb5kdc[3439](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786289735, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Aug 09 15:35:35 dist-test-slave-2w3w krb5kdc[3439](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786289735, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.3.89.193@KRBTEST.COM
I20260809 15:35:35.149354 3996 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:35:35.141123 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33154 (local address 127.3.89.193:34645)
0809 15:35:35.141245 (+ 122us) server_negotiation.cc:206] Beginning negotiation
0809 15:35:35.141249 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 15:35:35.141371 (+ 122us) server_negotiation.cc:407] Connection header received
0809 15:35:35.141504 (+ 133us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 15:35:35.141507 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 15:35:35.141550 (+ 43us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 15:35:35.141663 (+ 113us) server_negotiation.cc:226] Negotiated authn=SASL
0809 15:35:35.142362 (+ 699us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:35:35.143002 (+ 640us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:35:35.143829 (+ 827us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:35:35.144055 (+ 226us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:35:35.144130 (+ 75us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:35:35.146473 (+ 2343us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 15:35:35.146476 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 15:35:35.146478 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 15:35:35.146500 (+ 22us) server_negotiation.cc:922] calling sasl_server_start()
0809 15:35:35.147921 (+ 1421us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 15:35:35.148354 (+ 433us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 15:35:35.148356 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 15:35:35.148357 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 15:35:35.148432 (+ 75us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 15:35:35.148677 (+ 245us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 15:35:35.148679 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 15:35:35.148681 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 15:35:35.148832 (+ 151us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 15:35:35.148928 (+ 96us) server_negotiation.cc:1035] Waiting for connection context
0809 15:35:35.149133 (+ 205us) server_negotiation.cc:299] Negotiation successful
0809 15:35:35.149231 (+ 98us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":41}
I20260809 15:35:35.150665 3431 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskm314YR/build/debug/bin/kudu with pid 3554
I20260809 15:35:35.159492 3431 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskm314YR/build/debug/bin/kudu with pid 3701
I20260809 15:35:35.167580 3431 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskm314YR/build/debug/bin/kudu with pid 3849
I20260809 15:35:35.174404 3431 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskm314YR/build/debug/bin/kudu with pid 3454
2026-08-09T15:35:35Z chronyd exiting
[ OK ] SecurityITest.TestAuthorizationOnListTablets (2327 ms)
[ RUN ] SecurityITest.TestAuthorizationOnChecksum
Loading random data
Initializing database '/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786289732823074-3431-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 15:35:35 dist-test-slave-2w3w krb5kdc[4003](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 15:35:35 dist-test-slave-2w3w krb5kdc[4003](info): set up 2 sockets
Aug 09 15:35:35 dist-test-slave-2w3w krb5kdc[4003](info): commencing operation
krb5kdc: starting...
W20260809 15:35:36.261230 3431 mini_kdc.cc:121] Time spent starting KDC: real 1.063s user 0.002s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786289732823074-3431-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786289732823074-3431-0/krb5kdc/test-user.keytab.
Aug 09 15:35:36 dist-test-slave-2w3w krb5kdc[4003](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786289736, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T15:35:36Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T15:35:36Z Disabled control of system clock
WARNING: no policy specified for kudu/127.3.89.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.3.89.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.3.89.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786289732823074-3431-0/krb5kdc/kudu_127.3.89.254.keytab.
Entry for principal kudu/127.3.89.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786289732823074-3431-0/krb5kdc/kudu_127.3.89.254.keytab.
I20260809 15:35:36.389374 3431 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskm314YR/build/debug/bin/kudu
/tmp/dist-test-taskm314YR/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786289732823074-3431-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786289732823074-3431-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786289732823074-3431-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786289732823074-3431-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.3.89.254:41925
--webserver_interface=127.3.89.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786289732823074-3431-0/krb5kdc/kudu_127.3.89.254.keytab
--principal=kudu/127.3.89.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.3.89.212:34853
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.3.89.254:41925
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786289732823074-3431-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786289732823074-3431-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786289732823074-3431-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 15:35:36.498262 4017 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 15:35:36.498553 4017 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 15:35:36.498627 4017 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 15:35:36.502662 4017 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 15:35:36.502751 4017 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 15:35:36.502789 4017 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 15:35:36.502831 4017 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 15:35:36.502868 4017 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 15:35:36.508093 4017 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.3.89.212:34853
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786289732823074-3431-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786289732823074-3431-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.3.89.254:41925
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786289732823074-3431-0/krb5kdc/kudu_127.3.89.254.keytab
--principal=kudu/127.3.89.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.3.89.254:41925
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786289732823074-3431-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.3.89.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.4017
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786289732823074-3431-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 09 Aug 2026 14:46:55 UTC on ccycloud.quasar-yswkmf.root.comops.site
build id 4576540
I20260809 15:35:36.509120 4017 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 15:35:36.510118 4017 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 15:35:36.518401 4025 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 15:35:36.518401 4023 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 15:35:36.518429 4022 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 15:35:36.518841 4017 server_base.cc:1034] running on GCE node
I20260809 15:35:36.519418 4017 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 15:35:36.520476 4017 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 15:35:36.521659 4017 hybrid_clock.cc:648] HybridClock initialized: now 1786289736521639 us; error 31 us; skew 500 ppm
Aug 09 15:35:36 dist-test-slave-2w3w krb5kdc[4003](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786289736, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.89.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 15:35:36.524420 4017 init.cc:377] Logged in from keytab as kudu/127.3.89.254@KRBTEST.COM (short username kudu)
I20260809 15:35:36.525350 4017 webserver.cc:460] Webserver started at http://127.3.89.254:35269/ using document root <none> and password file <none>
I20260809 15:35:36.525894 4017 fs_manager.cc:359] Metadata directory not provided
I20260809 15:35:36.525987 4017 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 15:35:36.526207 4017 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 15:35:36.528244 4017 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786289732823074-3431-0/minicluster-data/master-0/data/instance:
uuid: "c42b6c2cdb0d4a04801c5946d6f07aef"
format_stamp: "Formatted at 2026-08-09 15:35:36 on dist-test-slave-2w3w"
server_key: "1ac7b9972f89e81911daff5103645d46"
server_key_iv: "53294fced337bd9a807888c583c01d67"
server_key_version: "encryptionkey@0"
I20260809 15:35:36.528810 4017 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786289732823074-3431-0/minicluster-data/master-0/wal/instance:
uuid: "c42b6c2cdb0d4a04801c5946d6f07aef"
format_stamp: "Formatted at 2026-08-09 15:35:36 on dist-test-slave-2w3w"
server_key: "1ac7b9972f89e81911daff5103645d46"
server_key_iv: "53294fced337bd9a807888c583c01d67"
server_key_version: "encryptionkey@0"
I20260809 15:35:36.532397 4017 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.005s sys 0.000s
I20260809 15:35:36.534649 4032 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 15:35:36.535835 4017 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260809 15:35:36.535998 4017 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786289732823074-3431-0/minicluster-data/master-0/data,/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786289732823074-3431-0/minicluster-data/master-0/wal
uuid: "c42b6c2cdb0d4a04801c5946d6f07aef"
format_stamp: "Formatted at 2026-08-09 15:35:36 on dist-test-slave-2w3w"
server_key: "1ac7b9972f89e81911daff5103645d46"
server_key_iv: "53294fced337bd9a807888c583c01d67"
server_key_version: "encryptionkey@0"
I20260809 15:35:36.536113 4017 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786289732823074-3431-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786289732823074-3431-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786289732823074-3431-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 15:35:36.555380 4017 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 15:35:36.558137 4017 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 15:35:36.558351 4017 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 15:35:36.565358 4017 rpc_server.cc:307] RPC server started. Bound to: 127.3.89.254:41925
I20260809 15:35:36.565400 4089 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.3.89.254:41925 every 8 connection(s)
I20260809 15:35:36.566250 4017 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786289732823074-3431-0/minicluster-data/master-0/data/info.pb
I20260809 15:35:36.568866 4090 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 15:35:36.574322 4090 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c42b6c2cdb0d4a04801c5946d6f07aef: Bootstrap starting.
I20260809 15:35:36.575726 3431 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskm314YR/build/debug/bin/kudu as pid 4017
I20260809 15:35:36.575845 3431 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786289732823074-3431-0/minicluster-data/master-0/wal/instance
I20260809 15:35:36.576082 3431 external_mini_cluster.cc:1428] Setting key 30ed93bd05a3c2333bf0d57b294e776c
I20260809 15:35:36.576972 4090 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P c42b6c2cdb0d4a04801c5946d6f07aef: Neither blocks nor log segments found. Creating new log.
I20260809 15:35:36.577831 4090 log.cc:824] T 00000000000000000000000000000000 P c42b6c2cdb0d4a04801c5946d6f07aef: Log is configured to *not* fsync() on all Append() calls
I20260809 15:35:36.579739 4090 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c42b6c2cdb0d4a04801c5946d6f07aef: No bootstrap required, opened a new log
I20260809 15:35:36.583230 4090 raft_consensus.cc:348] T 00000000000000000000000000000000 P c42b6c2cdb0d4a04801c5946d6f07aef [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c42b6c2cdb0d4a04801c5946d6f07aef" member_type: VOTER last_known_addr { host: "127.3.89.254" port: 41925 } }
I20260809 15:35:36.583478 4090 raft_consensus.cc:374] T 00000000000000000000000000000000 P c42b6c2cdb0d4a04801c5946d6f07aef [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 15:35:36.583554 4090 raft_consensus.cc:729] T 00000000000000000000000000000000 P c42b6c2cdb0d4a04801c5946d6f07aef [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c42b6c2cdb0d4a04801c5946d6f07aef, State: Initialized, Role: FOLLOWER
I20260809 15:35:36.583997 4090 consensus_queue.cc:260] T 00000000000000000000000000000000 P c42b6c2cdb0d4a04801c5946d6f07aef [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c42b6c2cdb0d4a04801c5946d6f07aef" member_type: VOTER last_known_addr { host: "127.3.89.254" port: 41925 } }
I20260809 15:35:36.584156 4090 raft_consensus.cc:388] T 00000000000000000000000000000000 P c42b6c2cdb0d4a04801c5946d6f07aef [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 15:35:36.584272 4090 raft_consensus.cc:482] T 00000000000000000000000000000000 P c42b6c2cdb0d4a04801c5946d6f07aef [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 15:35:36.584384 4090 raft_consensus.cc:3037] T 00000000000000000000000000000000 P c42b6c2cdb0d4a04801c5946d6f07aef [term 0 FOLLOWER]: Advancing to term 1
Aug 09 15:35:36 dist-test-slave-2w3w krb5kdc[4003](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786289736, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.3.89.254@KRBTEST.COM
I20260809 15:35:36.585352 4090 raft_consensus.cc:504] T 00000000000000000000000000000000 P c42b6c2cdb0d4a04801c5946d6f07aef [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c42b6c2cdb0d4a04801c5946d6f07aef" member_type: VOTER last_known_addr { host: "127.3.89.254" port: 41925 } }
I20260809 15:35:36.585747 4090 leader_election.cc:302] T 00000000000000000000000000000000 P c42b6c2cdb0d4a04801c5946d6f07aef [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: c42b6c2cdb0d4a04801c5946d6f07aef; no voters:
I20260809 15:35:36.586107 4090 leader_election.cc:288] T 00000000000000000000000000000000 P c42b6c2cdb0d4a04801c5946d6f07aef [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 15:35:36.586182 4096 raft_consensus.cc:2781] T 00000000000000000000000000000000 P c42b6c2cdb0d4a04801c5946d6f07aef [term 1 FOLLOWER]: Leader election won for term 1
I20260809 15:35:36.586473 4096 raft_consensus.cc:686] T 00000000000000000000000000000000 P c42b6c2cdb0d4a04801c5946d6f07aef [term 1 LEADER]: Becoming Leader. State: Replica: c42b6c2cdb0d4a04801c5946d6f07aef, State: Running, Role: LEADER
I20260809 15:35:36.587354 4090 sys_catalog.cc:564] T 00000000000000000000000000000000 P c42b6c2cdb0d4a04801c5946d6f07aef [sys.catalog]: configured and running, proceeding with master startup.
I20260809 15:35:36.587679 4096 consensus_queue.cc:237] T 00000000000000000000000000000000 P c42b6c2cdb0d4a04801c5946d6f07aef [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c42b6c2cdb0d4a04801c5946d6f07aef" member_type: VOTER last_known_addr { host: "127.3.89.254" port: 41925 } }
I20260809 15:35:36.589339 4097 sys_catalog.cc:455] T 00000000000000000000000000000000 P c42b6c2cdb0d4a04801c5946d6f07aef [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "c42b6c2cdb0d4a04801c5946d6f07aef" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c42b6c2cdb0d4a04801c5946d6f07aef" member_type: VOTER last_known_addr { host: "127.3.89.254" port: 41925 } } }
I20260809 15:35:36.589480 4097 sys_catalog.cc:458] T 00000000000000000000000000000000 P c42b6c2cdb0d4a04801c5946d6f07aef [sys.catalog]: This master's current role is: LEADER
I20260809 15:35:36.589804 4098 sys_catalog.cc:455] T 00000000000000000000000000000000 P c42b6c2cdb0d4a04801c5946d6f07aef [sys.catalog]: SysCatalogTable state changed. Reason: New leader c42b6c2cdb0d4a04801c5946d6f07aef. Latest consensus state: current_term: 1 leader_uuid: "c42b6c2cdb0d4a04801c5946d6f07aef" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c42b6c2cdb0d4a04801c5946d6f07aef" member_type: VOTER last_known_addr { host: "127.3.89.254" port: 41925 } } }
I20260809 15:35:36.590092 4111 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 15:35:36.590197 4098 sys_catalog.cc:458] T 00000000000000000000000000000000 P c42b6c2cdb0d4a04801c5946d6f07aef [sys.catalog]: This master's current role is: LEADER
I20260809 15:35:36.591611 4094 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:35:36.577194 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59700 (local address 127.3.89.254:41925)
0809 15:35:36.577635 (+ 441us) server_negotiation.cc:206] Beginning negotiation
0809 15:35:36.577650 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0809 15:35:36.577683 (+ 33us) server_negotiation.cc:407] Connection header received
0809 15:35:36.578484 (+ 801us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 15:35:36.578510 (+ 26us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 15:35:36.578842 (+ 332us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 15:35:36.579215 (+ 373us) server_negotiation.cc:226] Negotiated authn=SASL
0809 15:35:36.580033 (+ 818us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:35:36.581010 (+ 977us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:35:36.581867 (+ 857us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:35:36.582342 (+ 475us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:35:36.582506 (+ 164us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:35:36.585297 (+ 2791us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 15:35:36.585302 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 15:35:36.585314 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 15:35:36.585340 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0809 15:35:36.587467 (+ 2127us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 15:35:36.588015 (+ 548us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 15:35:36.588021 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 15:35:36.588028 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0809 15:35:36.588170 (+ 142us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 15:35:36.588560 (+ 390us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 15:35:36.588565 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 15:35:36.588566 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 15:35:36.589093 (+ 527us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 15:35:36.590179 (+ 1086us) server_negotiation.cc:1035] Waiting for connection context
0809 15:35:36.590342 (+ 163us) server_negotiation.cc:299] Negotiation successful
0809 15:35:36.590633 (+ 291us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":302,"thread_start_us":136,"threads_started":1}
I20260809 15:35:36.592955 4111 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 15:35:36.598579 4111 catalog_manager.cc:1347] Generated new cluster ID: a6a9496f8538424abda7cb1edbdb2705
I20260809 15:35:36.598654 4111 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 15:35:36.608423 4111 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 15:35:36.609465 4111 catalog_manager.cc:1504] Loading token signing keys...
I20260809 15:35:36.616614 4111 catalog_manager.cc:5868] T 00000000000000000000000000000000 P c42b6c2cdb0d4a04801c5946d6f07aef: Generated new TSK 0
I20260809 15:35:36.617386 4111 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.3.89.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.3.89.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.3.89.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786289732823074-3431-0/krb5kdc/kudu_127.3.89.193.keytab.
Entry for principal kudu/127.3.89.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786289732823074-3431-0/krb5kdc/kudu_127.3.89.193.keytab.
I20260809 15:35:36.655423 3431 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskm314YR/build/debug/bin/kudu
/tmp/dist-test-taskm314YR/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786289732823074-3431-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786289732823074-3431-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786289732823074-3431-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786289732823074-3431-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.3.89.193:0
--local_ip_for_outbound_sockets=127.3.89.193
--webserver_interface=127.3.89.193
--webserver_port=0
--tserver_master_addrs=127.3.89.254:41925
--keytab_file=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786289732823074-3431-0/krb5kdc/kudu_127.3.89.193.keytab
--principal=kudu/127.3.89.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.3.89.212:34853
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786289732823074-3431-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786289732823074-3431-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786289732823074-3431-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 15:35:36.768926 4120 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 15:35:36.769174 4120 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 15:35:36.769235 4120 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 15:35:36.773370 4120 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 15:35:36.773475 4120 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 15:35:36.773579 4120 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.3.89.193
I20260809 15:35:36.778753 4120 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.3.89.212:34853
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786289732823074-3431-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786289732823074-3431-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786289732823074-3431-0/krb5kdc/kudu_127.3.89.193.keytab
--principal=kudu/127.3.89.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.3.89.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786289732823074-3431-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.3.89.193
--webserver_port=0
--tserver_master_addrs=127.3.89.254:41925
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.4120
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.3.89.193
--log_dir=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786289732823074-3431-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 09 Aug 2026 14:46:55 UTC on ccycloud.quasar-yswkmf.root.comops.site
build id 4576540
I20260809 15:35:36.779780 4120 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 15:35:36.780810 4120 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 15:35:36.790727 4125 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 15:35:36.790812 4127 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 15:35:36.790738 4129 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 15:35:36.791220 4120 server_base.cc:1034] running on GCE node
I20260809 15:35:36.791615 4120 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 15:35:36.792223 4120 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 15:35:36.793377 4120 hybrid_clock.cc:648] HybridClock initialized: now 1786289736793348 us; error 43 us; skew 500 ppm
Aug 09 15:35:36 dist-test-slave-2w3w krb5kdc[4003](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786289736, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.89.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 15:35:36.796203 4120 init.cc:377] Logged in from keytab as kudu/127.3.89.193@KRBTEST.COM (short username kudu)
I20260809 15:35:36.797034 4120 webserver.cc:460] Webserver started at http://127.3.89.193:43343/ using document root <none> and password file <none>
I20260809 15:35:36.797539 4120 fs_manager.cc:359] Metadata directory not provided
I20260809 15:35:36.797600 4120 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 15:35:36.797753 4120 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 15:35:36.799986 4120 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786289732823074-3431-0/minicluster-data/ts-0/data/instance:
uuid: "1c5a51a59f73414aa719a4afede261a0"
format_stamp: "Formatted at 2026-08-09 15:35:36 on dist-test-slave-2w3w"
server_key: "239ef0ba2a9207c469f990713fae2755"
server_key_iv: "ad86e19b9681fcd5ba2e57fe429d7c7e"
server_key_version: "encryptionkey@0"
I20260809 15:35:36.800515 4120 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786289732823074-3431-0/minicluster-data/ts-0/wal/instance:
uuid: "1c5a51a59f73414aa719a4afede261a0"
format_stamp: "Formatted at 2026-08-09 15:35:36 on dist-test-slave-2w3w"
server_key: "239ef0ba2a9207c469f990713fae2755"
server_key_iv: "ad86e19b9681fcd5ba2e57fe429d7c7e"
server_key_version: "encryptionkey@0"
I20260809 15:35:36.804122 4120 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.004s sys 0.000s
I20260809 15:35:36.806557 4136 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 15:35:36.807811 4120 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260809 15:35:36.807941 4120 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786289732823074-3431-0/minicluster-data/ts-0/data,/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786289732823074-3431-0/minicluster-data/ts-0/wal
uuid: "1c5a51a59f73414aa719a4afede261a0"
format_stamp: "Formatted at 2026-08-09 15:35:36 on dist-test-slave-2w3w"
server_key: "239ef0ba2a9207c469f990713fae2755"
server_key_iv: "ad86e19b9681fcd5ba2e57fe429d7c7e"
server_key_version: "encryptionkey@0"
I20260809 15:35:36.808049 4120 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786289732823074-3431-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786289732823074-3431-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786289732823074-3431-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 15:35:36.818535 4120 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 15:35:36.821452 4120 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 15:35:36.821648 4120 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 15:35:36.822218 4120 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 15:35:36.823086 4120 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 15:35:36.823154 4120 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 15:35:36.823217 4120 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 15:35:36.823256 4120 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 15:35:36.831702 4120 rpc_server.cc:307] RPC server started. Bound to: 127.3.89.193:39293
I20260809 15:35:36.831771 4261 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.3.89.193:39293 every 8 connection(s)
I20260809 15:35:36.832582 4120 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786289732823074-3431-0/minicluster-data/ts-0/data/info.pb
I20260809 15:35:36.841674 3431 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskm314YR/build/debug/bin/kudu as pid 4120
I20260809 15:35:36.841766 3431 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786289732823074-3431-0/minicluster-data/ts-0/wal/instance
I20260809 15:35:36.842013 3431 external_mini_cluster.cc:1428] Setting key 09b4da9000b82dee43d3ba5b15840d7f
Aug 09 15:35:36 dist-test-slave-2w3w krb5kdc[4003](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786289736, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.89.193@KRBTEST.COM for kudu/127.3.89.254@KRBTEST.COM
I20260809 15:35:36.846992 4094 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:35:36.834359 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.89.193:55109 (local address 127.3.89.254:41925)
0809 15:35:36.834521 (+ 162us) server_negotiation.cc:206] Beginning negotiation
0809 15:35:36.834526 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 15:35:36.835351 (+ 825us) server_negotiation.cc:407] Connection header received
0809 15:35:36.836663 (+ 1312us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 15:35:36.836666 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 15:35:36.836714 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 15:35:36.836793 (+ 79us) server_negotiation.cc:226] Negotiated authn=SASL
0809 15:35:36.837936 (+ 1143us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:35:36.838489 (+ 553us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:35:36.839858 (+ 1369us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:35:36.840193 (+ 335us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:35:36.840374 (+ 181us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:35:36.843263 (+ 2889us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 15:35:36.843279 (+ 16us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 15:35:36.843285 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 15:35:36.843322 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0809 15:35:36.844748 (+ 1426us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 15:35:36.845414 (+ 666us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 15:35:36.845417 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 15:35:36.845418 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 15:35:36.845498 (+ 80us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 15:35:36.845874 (+ 376us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 15:35:36.845877 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 15:35:36.845879 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 15:35:36.846058 (+ 179us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 15:35:36.846165 (+ 107us) server_negotiation.cc:1035] Waiting for connection context
0809 15:35:36.846653 (+ 488us) server_negotiation.cc:299] Negotiation successful
0809 15:35:36.846794 (+ 141us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":60}
I20260809 15:35:36.847733 4264 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:35:36.834713 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.89.254:41925 (local address 127.3.89.193:55109)
0809 15:35:36.835190 (+ 477us) negotiation.cc:107] Waiting for socket to connect
0809 15:35:36.835237 (+ 47us) client_negotiation.cc:174] Beginning negotiation
0809 15:35:36.836463 (+ 1226us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 15:35:36.836985 (+ 522us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 15:35:36.836992 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 15:35:36.837334 (+ 342us) client_negotiation.cc:189] Negotiated authn=SASL
0809 15:35:36.837792 (+ 458us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:35:36.837804 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:35:36.838602 (+ 798us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:35:36.838604 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:35:36.839694 (+ 1090us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:35:36.839707 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:35:36.840427 (+ 720us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:35:36.840432 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:35:36.840613 (+ 181us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:35:36.841188 (+ 575us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 15:35:36.841205 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0809 15:35:36.843084 (+ 1879us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 15:35:36.844865 (+ 1781us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 15:35:36.844874 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 15:35:36.844890 (+ 16us) client_negotiation.cc:760] calling sasl_client_step()
0809 15:35:36.845284 (+ 394us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 15:35:36.845609 (+ 325us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 15:35:36.845612 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 15:35:36.845614 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 15:35:36.845763 (+ 149us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 15:35:36.846163 (+ 400us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 15:35:36.846170 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 15:35:36.846407 (+ 237us) client_negotiation.cc:769] Sending connection context
0809 15:35:36.846596 (+ 189us) client_negotiation.cc:240] Negotiation successful
0809 15:35:36.846915 (+ 319us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":278,"thread_start_us":163,"threads_started":1}
I20260809 15:35:36.848976 4262 heartbeater.cc:347] Connected to a master server at 127.3.89.254:41925
I20260809 15:35:36.849237 4262 heartbeater.cc:464] Registering TS with master...
I20260809 15:35:36.849840 4262 heartbeater.cc:511] Master 127.3.89.254:41925 requested a full tablet report, sending...
I20260809 15:35:36.851446 4051 ts_manager.cc:194] Registered new tserver with Master: 1c5a51a59f73414aa719a4afede261a0 (127.3.89.193:39293)
WARNING: no policy specified for kudu/127.3.89.194@KRBTEST.COM; defaulting to no policy
I20260809 15:35:36.852556 4051 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.3.89.193@KRBTEST.COM'} at 127.3.89.193:55109
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.3.89.194@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.3.89.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786289732823074-3431-0/krb5kdc/kudu_127.3.89.194.keytab.
Entry for principal kudu/127.3.89.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786289732823074-3431-0/krb5kdc/kudu_127.3.89.194.keytab.
I20260809 15:35:36.870314 3431 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskm314YR/build/debug/bin/kudu
/tmp/dist-test-taskm314YR/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786289732823074-3431-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786289732823074-3431-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786289732823074-3431-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786289732823074-3431-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.3.89.194:0
--local_ip_for_outbound_sockets=127.3.89.194
--webserver_interface=127.3.89.194
--webserver_port=0
--tserver_master_addrs=127.3.89.254:41925
--keytab_file=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786289732823074-3431-0/krb5kdc/kudu_127.3.89.194.keytab
--principal=kudu/127.3.89.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.3.89.212:34853
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786289732823074-3431-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786289732823074-3431-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786289732823074-3431-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 15:35:36.978791 4267 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 15:35:36.979068 4267 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 15:35:36.979142 4267 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 15:35:36.983050 4267 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 15:35:36.983103 4267 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 15:35:36.983193 4267 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.3.89.194
I20260809 15:35:36.988452 4267 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.3.89.212:34853
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786289732823074-3431-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786289732823074-3431-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786289732823074-3431-0/krb5kdc/kudu_127.3.89.194.keytab
--principal=kudu/127.3.89.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.3.89.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786289732823074-3431-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.3.89.194
--webserver_port=0
--tserver_master_addrs=127.3.89.254:41925
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.4267
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.3.89.194
--log_dir=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786289732823074-3431-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 09 Aug 2026 14:46:55 UTC on ccycloud.quasar-yswkmf.root.comops.site
build id 4576540
I20260809 15:35:36.989529 4267 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 15:35:36.990511 4267 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 15:35:37.000038 4273 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 15:35:37.000102 4272 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 15:35:37.000137 4275 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 15:35:37.000243 4267 server_base.cc:1034] running on GCE node
I20260809 15:35:37.000640 4267 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 15:35:37.001271 4267 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 15:35:37.002449 4267 hybrid_clock.cc:648] HybridClock initialized: now 1786289737002419 us; error 43 us; skew 500 ppm
Aug 09 15:35:37 dist-test-slave-2w3w krb5kdc[4003](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786289737, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.89.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 15:35:37.005296 4267 init.cc:377] Logged in from keytab as kudu/127.3.89.194@KRBTEST.COM (short username kudu)
I20260809 15:35:37.006148 4267 webserver.cc:460] Webserver started at http://127.3.89.194:44185/ using document root <none> and password file <none>
I20260809 15:35:37.006705 4267 fs_manager.cc:359] Metadata directory not provided
I20260809 15:35:37.006793 4267 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 15:35:37.007069 4267 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 15:35:37.009058 4267 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786289732823074-3431-0/minicluster-data/ts-1/data/instance:
uuid: "f65c01d090d9479fb7afeda5bdf5539b"
format_stamp: "Formatted at 2026-08-09 15:35:37 on dist-test-slave-2w3w"
server_key: "5787a8dee1e2b3013f2d6daf5e35b27d"
server_key_iv: "3602ab76fc1313b5a00ca83e3acf1956"
server_key_version: "encryptionkey@0"
I20260809 15:35:37.009675 4267 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786289732823074-3431-0/minicluster-data/ts-1/wal/instance:
uuid: "f65c01d090d9479fb7afeda5bdf5539b"
format_stamp: "Formatted at 2026-08-09 15:35:37 on dist-test-slave-2w3w"
server_key: "5787a8dee1e2b3013f2d6daf5e35b27d"
server_key_iv: "3602ab76fc1313b5a00ca83e3acf1956"
server_key_version: "encryptionkey@0"
I20260809 15:35:37.013314 4267 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.005s sys 0.000s
I20260809 15:35:37.015677 4283 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 15:35:37.016913 4267 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260809 15:35:37.017043 4267 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786289732823074-3431-0/minicluster-data/ts-1/data,/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786289732823074-3431-0/minicluster-data/ts-1/wal
uuid: "f65c01d090d9479fb7afeda5bdf5539b"
format_stamp: "Formatted at 2026-08-09 15:35:37 on dist-test-slave-2w3w"
server_key: "5787a8dee1e2b3013f2d6daf5e35b27d"
server_key_iv: "3602ab76fc1313b5a00ca83e3acf1956"
server_key_version: "encryptionkey@0"
I20260809 15:35:37.017150 4267 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786289732823074-3431-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786289732823074-3431-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786289732823074-3431-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 15:35:37.045310 4267 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 15:35:37.049669 4267 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 15:35:37.049870 4267 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 15:35:37.050598 4267 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 15:35:37.051761 4267 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 15:35:37.051824 4267 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 15:35:37.051898 4267 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 15:35:37.051934 4267 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 15:35:37.061317 4267 rpc_server.cc:307] RPC server started. Bound to: 127.3.89.194:37725
I20260809 15:35:37.061344 4407 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.3.89.194:37725 every 8 connection(s)
I20260809 15:35:37.062170 4267 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786289732823074-3431-0/minicluster-data/ts-1/data/info.pb
I20260809 15:35:37.067092 3431 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskm314YR/build/debug/bin/kudu as pid 4267
I20260809 15:35:37.067232 3431 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786289732823074-3431-0/minicluster-data/ts-1/wal/instance
I20260809 15:35:37.067517 3431 external_mini_cluster.cc:1428] Setting key 7dad82f4cbc8992b15074785741f9857
Aug 09 15:35:37 dist-test-slave-2w3w krb5kdc[4003](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786289737, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.89.194@KRBTEST.COM for kudu/127.3.89.254@KRBTEST.COM
I20260809 15:35:37.076506 4094 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:35:37.063940 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.89.194:41571 (local address 127.3.89.254:41925)
0809 15:35:37.064167 (+ 227us) server_negotiation.cc:206] Beginning negotiation
0809 15:35:37.064171 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 15:35:37.064847 (+ 676us) server_negotiation.cc:407] Connection header received
0809 15:35:37.066115 (+ 1268us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 15:35:37.066118 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 15:35:37.066165 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 15:35:37.066245 (+ 80us) server_negotiation.cc:226] Negotiated authn=SASL
0809 15:35:37.067518 (+ 1273us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:35:37.068238 (+ 720us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:35:37.069584 (+ 1346us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:35:37.069964 (+ 380us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:35:37.070088 (+ 124us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:35:37.072827 (+ 2739us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 15:35:37.072830 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 15:35:37.072833 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 15:35:37.072855 (+ 22us) server_negotiation.cc:922] calling sasl_server_start()
0809 15:35:37.074429 (+ 1574us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 15:35:37.074952 (+ 523us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 15:35:37.074990 (+ 38us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 15:35:37.074992 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 15:35:37.075074 (+ 82us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 15:35:37.075414 (+ 340us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 15:35:37.075418 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 15:35:37.075419 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 15:35:37.075592 (+ 173us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 15:35:37.075694 (+ 102us) server_negotiation.cc:1035] Waiting for connection context
0809 15:35:37.076192 (+ 498us) server_negotiation.cc:299] Negotiation successful
0809 15:35:37.076336 (+ 144us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":133}
I20260809 15:35:37.077256 4410 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:35:37.064273 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.89.254:41925 (local address 127.3.89.194:41571)
0809 15:35:37.064698 (+ 425us) negotiation.cc:107] Waiting for socket to connect
0809 15:35:37.064740 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0809 15:35:37.065912 (+ 1172us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 15:35:37.066392 (+ 480us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 15:35:37.066398 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 15:35:37.066740 (+ 342us) client_negotiation.cc:189] Negotiated authn=SASL
0809 15:35:37.067361 (+ 621us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:35:37.067384 (+ 23us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:35:37.068369 (+ 985us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:35:37.068373 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:35:37.069459 (+ 1086us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:35:37.069470 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:35:37.070103 (+ 633us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:35:37.070106 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:35:37.070242 (+ 136us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:35:37.070820 (+ 578us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 15:35:37.070835 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0809 15:35:37.072668 (+ 1833us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 15:35:37.074543 (+ 1875us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 15:35:37.074549 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 15:35:37.074562 (+ 13us) client_negotiation.cc:760] calling sasl_client_step()
0809 15:35:37.074838 (+ 276us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 15:35:37.075165 (+ 327us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 15:35:37.075168 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 15:35:37.075170 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 15:35:37.075311 (+ 141us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 15:35:37.075692 (+ 381us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 15:35:37.075699 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 15:35:37.075945 (+ 246us) client_negotiation.cc:769] Sending connection context
0809 15:35:37.076148 (+ 203us) client_negotiation.cc:240] Negotiation successful
0809 15:35:37.076456 (+ 308us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":248,"thread_start_us":115,"threads_started":1}
WARNING: no policy specified for kudu/127.3.89.195@KRBTEST.COM; defaulting to no policy
I20260809 15:35:37.078516 4408 heartbeater.cc:347] Connected to a master server at 127.3.89.254:41925
I20260809 15:35:37.078765 4408 heartbeater.cc:464] Registering TS with master...
I20260809 15:35:37.079351 4408 heartbeater.cc:511] Master 127.3.89.254:41925 requested a full tablet report, sending...
I20260809 15:35:37.080343 4051 ts_manager.cc:194] Registered new tserver with Master: f65c01d090d9479fb7afeda5bdf5539b (127.3.89.194:37725)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.3.89.195@KRBTEST.COM" created.
I20260809 15:35:37.080964 4051 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.3.89.194@KRBTEST.COM'} at 127.3.89.194:41571
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.3.89.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786289732823074-3431-0/krb5kdc/kudu_127.3.89.195.keytab.
Entry for principal kudu/127.3.89.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786289732823074-3431-0/krb5kdc/kudu_127.3.89.195.keytab.
I20260809 15:35:37.097210 3431 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskm314YR/build/debug/bin/kudu
/tmp/dist-test-taskm314YR/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786289732823074-3431-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786289732823074-3431-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786289732823074-3431-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786289732823074-3431-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.3.89.195:0
--local_ip_for_outbound_sockets=127.3.89.195
--webserver_interface=127.3.89.195
--webserver_port=0
--tserver_master_addrs=127.3.89.254:41925
--keytab_file=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786289732823074-3431-0/krb5kdc/kudu_127.3.89.195.keytab
--principal=kudu/127.3.89.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.3.89.212:34853
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786289732823074-3431-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786289732823074-3431-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786289732823074-3431-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 15:35:37.207016 4413 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 15:35:37.207273 4413 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 15:35:37.207378 4413 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 15:35:37.211359 4413 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 15:35:37.211423 4413 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 15:35:37.211508 4413 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.3.89.195
I20260809 15:35:37.216645 4413 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.3.89.212:34853
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786289732823074-3431-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786289732823074-3431-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786289732823074-3431-0/krb5kdc/kudu_127.3.89.195.keytab
--principal=kudu/127.3.89.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.3.89.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786289732823074-3431-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.3.89.195
--webserver_port=0
--tserver_master_addrs=127.3.89.254:41925
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.4413
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.3.89.195
--log_dir=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786289732823074-3431-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 09 Aug 2026 14:46:55 UTC on ccycloud.quasar-yswkmf.root.comops.site
build id 4576540
I20260809 15:35:37.217643 4413 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 15:35:37.218616 4413 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 15:35:37.227851 4420 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 15:35:37.227926 4423 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 15:35:37.227851 4419 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 15:35:37.228078 4413 server_base.cc:1034] running on GCE node
I20260809 15:35:37.228608 4413 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 15:35:37.229238 4413 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 15:35:37.230409 4413 hybrid_clock.cc:648] HybridClock initialized: now 1786289737230389 us; error 44 us; skew 500 ppm
Aug 09 15:35:37 dist-test-slave-2w3w krb5kdc[4003](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786289737, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.89.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 15:35:37.233024 4413 init.cc:377] Logged in from keytab as kudu/127.3.89.195@KRBTEST.COM (short username kudu)
I20260809 15:35:37.233804 4413 webserver.cc:460] Webserver started at http://127.3.89.195:33821/ using document root <none> and password file <none>
I20260809 15:35:37.234339 4413 fs_manager.cc:359] Metadata directory not provided
I20260809 15:35:37.234426 4413 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 15:35:37.234645 4413 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 15:35:37.236644 4413 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786289732823074-3431-0/minicluster-data/ts-2/data/instance:
uuid: "b8ccc17f0dd246cfb5f23f58781a2600"
format_stamp: "Formatted at 2026-08-09 15:35:37 on dist-test-slave-2w3w"
server_key: "cc7708b7b60698dcb077cff32287649b"
server_key_iv: "cf76cc5bca916a841fb679ad7e1dc048"
server_key_version: "encryptionkey@0"
I20260809 15:35:37.237200 4413 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786289732823074-3431-0/minicluster-data/ts-2/wal/instance:
uuid: "b8ccc17f0dd246cfb5f23f58781a2600"
format_stamp: "Formatted at 2026-08-09 15:35:37 on dist-test-slave-2w3w"
server_key: "cc7708b7b60698dcb077cff32287649b"
server_key_iv: "cf76cc5bca916a841fb679ad7e1dc048"
server_key_version: "encryptionkey@0"
I20260809 15:35:37.240862 4413 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.005s sys 0.000s
I20260809 15:35:37.243134 4430 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 15:35:37.244275 4413 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260809 15:35:37.244403 4413 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786289732823074-3431-0/minicluster-data/ts-2/data,/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786289732823074-3431-0/minicluster-data/ts-2/wal
uuid: "b8ccc17f0dd246cfb5f23f58781a2600"
format_stamp: "Formatted at 2026-08-09 15:35:37 on dist-test-slave-2w3w"
server_key: "cc7708b7b60698dcb077cff32287649b"
server_key_iv: "cf76cc5bca916a841fb679ad7e1dc048"
server_key_version: "encryptionkey@0"
I20260809 15:35:37.244511 4413 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786289732823074-3431-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786289732823074-3431-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786289732823074-3431-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 15:35:37.256587 4413 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 15:35:37.259263 4413 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 15:35:37.259451 4413 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 15:35:37.259994 4413 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 15:35:37.260764 4413 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 15:35:37.260833 4413 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 15:35:37.260897 4413 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 15:35:37.260936 4413 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 15:35:37.269408 4413 rpc_server.cc:307] RPC server started. Bound to: 127.3.89.195:33345
I20260809 15:35:37.269450 4552 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.3.89.195:33345 every 8 connection(s)
I20260809 15:35:37.270233 4413 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786289732823074-3431-0/minicluster-data/ts-2/data/info.pb
I20260809 15:35:37.273708 3431 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskm314YR/build/debug/bin/kudu as pid 4413
I20260809 15:35:37.273859 3431 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786289732823074-3431-0/minicluster-data/ts-2/wal/instance
I20260809 15:35:37.274147 3431 external_mini_cluster.cc:1428] Setting key e65d229d9c2cb2f69a5de5d908ad4eb1
Aug 09 15:35:37 dist-test-slave-2w3w krb5kdc[4003](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786289737, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.89.195@KRBTEST.COM for kudu/127.3.89.254@KRBTEST.COM
I20260809 15:35:37.283736 4094 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:35:37.272053 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.89.195:38579 (local address 127.3.89.254:41925)
0809 15:35:37.272198 (+ 145us) server_negotiation.cc:206] Beginning negotiation
0809 15:35:37.272202 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 15:35:37.272998 (+ 796us) server_negotiation.cc:407] Connection header received
0809 15:35:37.274665 (+ 1667us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 15:35:37.274669 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 15:35:37.274736 (+ 67us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 15:35:37.274830 (+ 94us) server_negotiation.cc:226] Negotiated authn=SASL
0809 15:35:37.276076 (+ 1246us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:35:37.276611 (+ 535us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:35:37.277723 (+ 1112us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:35:37.277962 (+ 239us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:35:37.278072 (+ 110us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:35:37.280469 (+ 2397us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 15:35:37.280478 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 15:35:37.280480 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 15:35:37.280504 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0809 15:35:37.281748 (+ 1244us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 15:35:37.282233 (+ 485us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 15:35:37.282237 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 15:35:37.282239 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 15:35:37.282322 (+ 83us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 15:35:37.282646 (+ 324us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 15:35:37.282651 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 15:35:37.282652 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 15:35:37.282829 (+ 177us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 15:35:37.282935 (+ 106us) server_negotiation.cc:1035] Waiting for connection context
0809 15:35:37.283417 (+ 482us) server_negotiation.cc:299] Negotiation successful
0809 15:35:37.283534 (+ 117us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":52}
I20260809 15:35:37.284439 4555 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:35:37.272368 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.89.254:41925 (local address 127.3.89.195:38579)
0809 15:35:37.272838 (+ 470us) negotiation.cc:107] Waiting for socket to connect
0809 15:35:37.272879 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0809 15:35:37.274434 (+ 1555us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 15:35:37.275053 (+ 619us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 15:35:37.275059 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 15:35:37.275486 (+ 427us) client_negotiation.cc:189] Negotiated authn=SASL
0809 15:35:37.275934 (+ 448us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:35:37.275946 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:35:37.276734 (+ 788us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:35:37.276743 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:35:37.277600 (+ 857us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:35:37.277609 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:35:37.278108 (+ 499us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:35:37.278113 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:35:37.278294 (+ 181us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:35:37.278845 (+ 551us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 15:35:37.278861 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0809 15:35:37.280313 (+ 1452us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 15:35:37.281863 (+ 1550us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 15:35:37.281869 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 15:35:37.281881 (+ 12us) client_negotiation.cc:760] calling sasl_client_step()
0809 15:35:37.282129 (+ 248us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 15:35:37.282414 (+ 285us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 15:35:37.282418 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 15:35:37.282420 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 15:35:37.282552 (+ 132us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 15:35:37.282932 (+ 380us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 15:35:37.282938 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 15:35:37.283185 (+ 247us) client_negotiation.cc:769] Sending connection context
0809 15:35:37.283361 (+ 176us) client_negotiation.cc:240] Negotiation successful
0809 15:35:37.283696 (+ 335us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":285,"thread_start_us":126,"threads_started":1}
I20260809 15:35:37.285468 4553 heartbeater.cc:347] Connected to a master server at 127.3.89.254:41925
I20260809 15:35:37.285707 4553 heartbeater.cc:464] Registering TS with master...
I20260809 15:35:37.286186 4553 heartbeater.cc:511] Master 127.3.89.254:41925 requested a full tablet report, sending...
I20260809 15:35:37.287156 4051 ts_manager.cc:194] Registered new tserver with Master: b8ccc17f0dd246cfb5f23f58781a2600 (127.3.89.195:33345)
I20260809 15:35:37.287810 4051 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.3.89.195@KRBTEST.COM'} at 127.3.89.195:38579
I20260809 15:35:37.287851 3431 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260809 15:35:37.299036 4094 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:35:37.292313 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59706 (local address 127.3.89.254:41925)
0809 15:35:37.292429 (+ 116us) server_negotiation.cc:206] Beginning negotiation
0809 15:35:37.292433 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 15:35:37.292599 (+ 166us) server_negotiation.cc:407] Connection header received
0809 15:35:37.292682 (+ 83us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 15:35:37.292684 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 15:35:37.292719 (+ 35us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 15:35:37.292786 (+ 67us) server_negotiation.cc:226] Negotiated authn=SASL
0809 15:35:37.293423 (+ 637us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:35:37.293894 (+ 471us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:35:37.294658 (+ 764us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:35:37.294863 (+ 205us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:35:37.294937 (+ 74us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:35:37.296173 (+ 1236us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 15:35:37.296176 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 15:35:37.296177 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 15:35:37.296197 (+ 20us) server_negotiation.cc:922] calling sasl_server_start()
0809 15:35:37.297574 (+ 1377us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 15:35:37.297987 (+ 413us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 15:35:37.297990 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 15:35:37.297991 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 15:35:37.298060 (+ 69us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 15:35:37.298289 (+ 229us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 15:35:37.298291 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 15:35:37.298292 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 15:35:37.298479 (+ 187us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 15:35:37.298581 (+ 102us) server_negotiation.cc:1035] Waiting for connection context
0809 15:35:37.298791 (+ 210us) server_negotiation.cc:299] Negotiation successful
0809 15:35:37.298874 (+ 83us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":52}
I20260809 15:35:37.304349 4051 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:59706:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260809 15:35:37.307628 4051 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260809 15:35:37.322821 4570 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:35:37.318108 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55170 (local address 127.3.89.194:37725)
0809 15:35:37.318463 (+ 355us) server_negotiation.cc:206] Beginning negotiation
0809 15:35:37.318476 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0809 15:35:37.318497 (+ 21us) server_negotiation.cc:407] Connection header received
0809 15:35:37.318572 (+ 75us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 15:35:37.318580 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 15:35:37.318736 (+ 156us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 15:35:37.318878 (+ 142us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 15:35:37.319169 (+ 291us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:35:37.319992 (+ 823us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:35:37.321577 (+ 1585us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:35:37.322131 (+ 554us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:35:37.322248 (+ 117us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:35:37.322456 (+ 208us) server_negotiation.cc:1035] Waiting for connection context
0809 15:35:37.322576 (+ 120us) server_negotiation.cc:299] Negotiation successful
0809 15:35:37.322662 (+ 86us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":261,"thread_start_us":145,"threads_started":1}
I20260809 15:35:37.322813 4565 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:35:37.316858 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.89.194:37725 (local address 127.0.0.1:55170)
0809 15:35:37.317279 (+ 421us) negotiation.cc:107] Waiting for socket to connect
0809 15:35:37.317293 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0809 15:35:37.317392 (+ 99us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 15:35:37.318852 (+ 1460us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 15:35:37.318858 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 15:35:37.318871 (+ 13us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 15:35:37.319014 (+ 143us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:35:37.319022 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:35:37.320129 (+ 1107us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:35:37.320134 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:35:37.321464 (+ 1330us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:35:37.321473 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:35:37.322248 (+ 775us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:35:37.322252 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:35:37.322392 (+ 140us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:35:37.322432 (+ 40us) client_negotiation.cc:769] Sending connection context
0809 15:35:37.322548 (+ 116us) client_negotiation.cc:240] Negotiation successful
0809 15:35:37.322662 (+ 114us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":360,"mutex_wait_us":53,"thread_start_us":98,"threads_started":1}
I20260809 15:35:37.323392 4564 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:35:37.316781 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.89.195:33345 (local address 127.0.0.1:50650)
0809 15:35:37.317091 (+ 310us) negotiation.cc:107] Waiting for socket to connect
0809 15:35:37.317134 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0809 15:35:37.317250 (+ 116us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 15:35:37.318719 (+ 1469us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 15:35:37.318721 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 15:35:37.318734 (+ 13us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 15:35:37.319008 (+ 274us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:35:37.319019 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:35:37.320535 (+ 1516us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:35:37.320538 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:35:37.321799 (+ 1261us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:35:37.321809 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:35:37.322917 (+ 1108us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:35:37.322923 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:35:37.323080 (+ 157us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:35:37.323107 (+ 27us) client_negotiation.cc:769] Sending connection context
0809 15:35:37.323163 (+ 56us) client_negotiation.cc:240] Negotiation successful
0809 15:35:37.323239 (+ 76us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":156,"mutex_wait_us":75,"thread_start_us":87,"threads_started":1}
I20260809 15:35:37.324014 4567 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:35:37.317467 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50650 (local address 127.3.89.195:33345)
0809 15:35:37.318299 (+ 832us) server_negotiation.cc:206] Beginning negotiation
0809 15:35:37.318312 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0809 15:35:37.318335 (+ 23us) server_negotiation.cc:407] Connection header received
0809 15:35:37.318408 (+ 73us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 15:35:37.318419 (+ 11us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 15:35:37.318556 (+ 137us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 15:35:37.318926 (+ 370us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 15:35:37.319128 (+ 202us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:35:37.320378 (+ 1250us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:35:37.321952 (+ 1574us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:35:37.322766 (+ 814us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:35:37.323569 (+ 803us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:35:37.323708 (+ 139us) server_negotiation.cc:1035] Waiting for connection context
0809 15:35:37.323759 (+ 51us) server_negotiation.cc:299] Negotiation successful
0809 15:35:37.323846 (+ 87us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":713,"thread_start_us":170,"threads_started":1}
I20260809 15:35:37.324689 4566 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:35:37.317506 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.89.193:39293 (local address 127.0.0.1:34730)
0809 15:35:37.317723 (+ 217us) negotiation.cc:107] Waiting for socket to connect
0809 15:35:37.317736 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0809 15:35:37.317842 (+ 106us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 15:35:37.318674 (+ 832us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 15:35:37.318681 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 15:35:37.318707 (+ 26us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 15:35:37.319234 (+ 527us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:35:37.319243 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:35:37.320684 (+ 1441us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:35:37.320689 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:35:37.321923 (+ 1234us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:35:37.321931 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:35:37.324281 (+ 2350us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:35:37.324285 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:35:37.324369 (+ 84us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:35:37.324390 (+ 21us) client_negotiation.cc:769] Sending connection context
0809 15:35:37.324492 (+ 102us) client_negotiation.cc:240] Negotiation successful
0809 15:35:37.324567 (+ 75us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":158,"mutex_wait_us":401,"thread_start_us":85,"threads_started":1}
I20260809 15:35:37.324792 4569 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:35:37.317810 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34730 (local address 127.3.89.193:39293)
0809 15:35:37.318151 (+ 341us) server_negotiation.cc:206] Beginning negotiation
0809 15:35:37.318171 (+ 20us) server_negotiation.cc:399] Waiting for connection header
0809 15:35:37.318195 (+ 24us) server_negotiation.cc:407] Connection header received
0809 15:35:37.318293 (+ 98us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 15:35:37.318299 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 15:35:37.318502 (+ 203us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 15:35:37.318660 (+ 158us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 15:35:37.319336 (+ 676us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:35:37.320548 (+ 1212us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:35:37.322059 (+ 1511us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:35:37.322829 (+ 770us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:35:37.324270 (+ 1441us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:35:37.324406 (+ 136us) server_negotiation.cc:1035] Waiting for connection context
0809 15:35:37.324548 (+ 142us) server_negotiation.cc:299] Negotiation successful
0809 15:35:37.324636 (+ 88us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":234,"thread_start_us":144,"threads_started":1}
I20260809 15:35:37.326242 4336 tablet_service.cc:1467] Processing CreateTablet for tablet b88f8fc9048e421eab27392b7d0d4322 (DEFAULT_TABLE table=test-table [id=93c3b0baba3e41a298a00988219d21b8]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 15:35:37.327585 4336 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b88f8fc9048e421eab27392b7d0d4322. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 15:35:37.327333 4189 tablet_service.cc:1467] Processing CreateTablet for tablet b88f8fc9048e421eab27392b7d0d4322 (DEFAULT_TABLE table=test-table [id=93c3b0baba3e41a298a00988219d21b8]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 15:35:37.328284 4189 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b88f8fc9048e421eab27392b7d0d4322. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 15:35:37.332940 4482 tablet_service.cc:1467] Processing CreateTablet for tablet b88f8fc9048e421eab27392b7d0d4322 (DEFAULT_TABLE table=test-table [id=93c3b0baba3e41a298a00988219d21b8]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 15:35:37.333770 4572 tablet_bootstrap.cc:492] T b88f8fc9048e421eab27392b7d0d4322 P f65c01d090d9479fb7afeda5bdf5539b: Bootstrap starting.
I20260809 15:35:37.333931 4482 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b88f8fc9048e421eab27392b7d0d4322. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 15:35:37.335014 4573 tablet_bootstrap.cc:492] T b88f8fc9048e421eab27392b7d0d4322 P 1c5a51a59f73414aa719a4afede261a0: Bootstrap starting.
I20260809 15:35:37.336369 4572 tablet_bootstrap.cc:654] T b88f8fc9048e421eab27392b7d0d4322 P f65c01d090d9479fb7afeda5bdf5539b: Neither blocks nor log segments found. Creating new log.
I20260809 15:35:37.337070 4572 log.cc:824] T b88f8fc9048e421eab27392b7d0d4322 P f65c01d090d9479fb7afeda5bdf5539b: Log is configured to *not* fsync() on all Append() calls
I20260809 15:35:37.337114 4573 tablet_bootstrap.cc:654] T b88f8fc9048e421eab27392b7d0d4322 P 1c5a51a59f73414aa719a4afede261a0: Neither blocks nor log segments found. Creating new log.
I20260809 15:35:37.338491 4573 log.cc:824] T b88f8fc9048e421eab27392b7d0d4322 P 1c5a51a59f73414aa719a4afede261a0: Log is configured to *not* fsync() on all Append() calls
I20260809 15:35:37.339324 4572 tablet_bootstrap.cc:492] T b88f8fc9048e421eab27392b7d0d4322 P f65c01d090d9479fb7afeda5bdf5539b: No bootstrap required, opened a new log
I20260809 15:35:37.339524 4572 ts_tablet_manager.cc:1397] T b88f8fc9048e421eab27392b7d0d4322 P f65c01d090d9479fb7afeda5bdf5539b: Time spent bootstrapping tablet: real 0.006s user 0.004s sys 0.001s
I20260809 15:35:37.340504 4573 tablet_bootstrap.cc:492] T b88f8fc9048e421eab27392b7d0d4322 P 1c5a51a59f73414aa719a4afede261a0: No bootstrap required, opened a new log
I20260809 15:35:37.340693 4573 ts_tablet_manager.cc:1397] T b88f8fc9048e421eab27392b7d0d4322 P 1c5a51a59f73414aa719a4afede261a0: Time spent bootstrapping tablet: real 0.006s user 0.005s sys 0.000s
I20260809 15:35:37.341519 4576 tablet_bootstrap.cc:492] T b88f8fc9048e421eab27392b7d0d4322 P b8ccc17f0dd246cfb5f23f58781a2600: Bootstrap starting.
I20260809 15:35:37.342643 4572 raft_consensus.cc:348] T b88f8fc9048e421eab27392b7d0d4322 P f65c01d090d9479fb7afeda5bdf5539b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8ccc17f0dd246cfb5f23f58781a2600" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 33345 } } peers { permanent_uuid: "f65c01d090d9479fb7afeda5bdf5539b" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 37725 } } peers { permanent_uuid: "1c5a51a59f73414aa719a4afede261a0" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 39293 } }
I20260809 15:35:37.342875 4572 raft_consensus.cc:374] T b88f8fc9048e421eab27392b7d0d4322 P f65c01d090d9479fb7afeda5bdf5539b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 15:35:37.342926 4572 raft_consensus.cc:729] T b88f8fc9048e421eab27392b7d0d4322 P f65c01d090d9479fb7afeda5bdf5539b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f65c01d090d9479fb7afeda5bdf5539b, State: Initialized, Role: FOLLOWER
I20260809 15:35:37.343362 4572 consensus_queue.cc:260] T b88f8fc9048e421eab27392b7d0d4322 P f65c01d090d9479fb7afeda5bdf5539b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8ccc17f0dd246cfb5f23f58781a2600" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 33345 } } peers { permanent_uuid: "f65c01d090d9479fb7afeda5bdf5539b" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 37725 } } peers { permanent_uuid: "1c5a51a59f73414aa719a4afede261a0" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 39293 } }
I20260809 15:35:37.343612 4576 tablet_bootstrap.cc:654] T b88f8fc9048e421eab27392b7d0d4322 P b8ccc17f0dd246cfb5f23f58781a2600: Neither blocks nor log segments found. Creating new log.
I20260809 15:35:37.344079 4408 heartbeater.cc:503] Master 127.3.89.254:41925 was elected leader, sending a full tablet report...
I20260809 15:35:37.344584 4573 raft_consensus.cc:348] T b88f8fc9048e421eab27392b7d0d4322 P 1c5a51a59f73414aa719a4afede261a0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8ccc17f0dd246cfb5f23f58781a2600" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 33345 } } peers { permanent_uuid: "f65c01d090d9479fb7afeda5bdf5539b" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 37725 } } peers { permanent_uuid: "1c5a51a59f73414aa719a4afede261a0" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 39293 } }
I20260809 15:35:37.344830 4573 raft_consensus.cc:374] T b88f8fc9048e421eab27392b7d0d4322 P 1c5a51a59f73414aa719a4afede261a0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 15:35:37.344846 4576 log.cc:824] T b88f8fc9048e421eab27392b7d0d4322 P b8ccc17f0dd246cfb5f23f58781a2600: Log is configured to *not* fsync() on all Append() calls
I20260809 15:35:37.344882 4573 raft_consensus.cc:729] T b88f8fc9048e421eab27392b7d0d4322 P 1c5a51a59f73414aa719a4afede261a0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1c5a51a59f73414aa719a4afede261a0, State: Initialized, Role: FOLLOWER
I20260809 15:35:37.345317 4573 consensus_queue.cc:260] T b88f8fc9048e421eab27392b7d0d4322 P 1c5a51a59f73414aa719a4afede261a0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8ccc17f0dd246cfb5f23f58781a2600" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 33345 } } peers { permanent_uuid: "f65c01d090d9479fb7afeda5bdf5539b" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 37725 } } peers { permanent_uuid: "1c5a51a59f73414aa719a4afede261a0" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 39293 } }
I20260809 15:35:37.345795 4572 ts_tablet_manager.cc:1428] T b88f8fc9048e421eab27392b7d0d4322 P f65c01d090d9479fb7afeda5bdf5539b: Time spent starting tablet: real 0.006s user 0.003s sys 0.002s
I20260809 15:35:37.346316 4262 heartbeater.cc:503] Master 127.3.89.254:41925 was elected leader, sending a full tablet report...
I20260809 15:35:37.347010 4573 ts_tablet_manager.cc:1428] T b88f8fc9048e421eab27392b7d0d4322 P 1c5a51a59f73414aa719a4afede261a0: Time spent starting tablet: real 0.006s user 0.006s sys 0.000s
I20260809 15:35:37.347241 4576 tablet_bootstrap.cc:492] T b88f8fc9048e421eab27392b7d0d4322 P b8ccc17f0dd246cfb5f23f58781a2600: No bootstrap required, opened a new log
I20260809 15:35:37.347540 4576 ts_tablet_manager.cc:1397] T b88f8fc9048e421eab27392b7d0d4322 P b8ccc17f0dd246cfb5f23f58781a2600: Time spent bootstrapping tablet: real 0.006s user 0.005s sys 0.000s
I20260809 15:35:37.350899 4576 raft_consensus.cc:348] T b88f8fc9048e421eab27392b7d0d4322 P b8ccc17f0dd246cfb5f23f58781a2600 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8ccc17f0dd246cfb5f23f58781a2600" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 33345 } } peers { permanent_uuid: "f65c01d090d9479fb7afeda5bdf5539b" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 37725 } } peers { permanent_uuid: "1c5a51a59f73414aa719a4afede261a0" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 39293 } }
I20260809 15:35:37.351162 4576 raft_consensus.cc:374] T b88f8fc9048e421eab27392b7d0d4322 P b8ccc17f0dd246cfb5f23f58781a2600 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 15:35:37.351235 4576 raft_consensus.cc:729] T b88f8fc9048e421eab27392b7d0d4322 P b8ccc17f0dd246cfb5f23f58781a2600 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b8ccc17f0dd246cfb5f23f58781a2600, State: Initialized, Role: FOLLOWER
I20260809 15:35:37.351609 4576 consensus_queue.cc:260] T b88f8fc9048e421eab27392b7d0d4322 P b8ccc17f0dd246cfb5f23f58781a2600 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8ccc17f0dd246cfb5f23f58781a2600" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 33345 } } peers { permanent_uuid: "f65c01d090d9479fb7afeda5bdf5539b" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 37725 } } peers { permanent_uuid: "1c5a51a59f73414aa719a4afede261a0" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 39293 } }
I20260809 15:35:37.352173 4553 heartbeater.cc:503] Master 127.3.89.254:41925 was elected leader, sending a full tablet report...
I20260809 15:35:37.352435 4576 ts_tablet_manager.cc:1428] T b88f8fc9048e421eab27392b7d0d4322 P b8ccc17f0dd246cfb5f23f58781a2600: Time spent starting tablet: real 0.005s user 0.004s sys 0.000s
W20260809 15:35:37.521255 4554 tablet.cc:2369] T b88f8fc9048e421eab27392b7d0d4322 P b8ccc17f0dd246cfb5f23f58781a2600: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260809 15:35:37.563215 4409 tablet.cc:2369] T b88f8fc9048e421eab27392b7d0d4322 P f65c01d090d9479fb7afeda5bdf5539b: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260809 15:35:37.583877 4263 tablet.cc:2369] T b88f8fc9048e421eab27392b7d0d4322 P 1c5a51a59f73414aa719a4afede261a0: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260809 15:35:37.628289 4577 raft_consensus.cc:482] T b88f8fc9048e421eab27392b7d0d4322 P f65c01d090d9479fb7afeda5bdf5539b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 15:35:37.628506 4577 raft_consensus.cc:504] T b88f8fc9048e421eab27392b7d0d4322 P f65c01d090d9479fb7afeda5bdf5539b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8ccc17f0dd246cfb5f23f58781a2600" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 33345 } } peers { permanent_uuid: "f65c01d090d9479fb7afeda5bdf5539b" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 37725 } } peers { permanent_uuid: "1c5a51a59f73414aa719a4afede261a0" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 39293 } }
I20260809 15:35:37.629689 4577 leader_election.cc:288] T b88f8fc9048e421eab27392b7d0d4322 P f65c01d090d9479fb7afeda5bdf5539b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b8ccc17f0dd246cfb5f23f58781a2600 (127.3.89.195:33345), 1c5a51a59f73414aa719a4afede261a0 (127.3.89.193:39293)
I20260809 15:35:37.635187 4569 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:35:37.630059 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.89.194:55555 (local address 127.3.89.193:39293)
0809 15:35:37.630189 (+ 130us) server_negotiation.cc:206] Beginning negotiation
0809 15:35:37.630194 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 15:35:37.630396 (+ 202us) server_negotiation.cc:407] Connection header received
0809 15:35:37.630517 (+ 121us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 15:35:37.630520 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 15:35:37.630569 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 15:35:37.630649 (+ 80us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 15:35:37.631119 (+ 470us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:35:37.631921 (+ 802us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:35:37.633883 (+ 1962us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:35:37.634492 (+ 609us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:35:37.634610 (+ 118us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:35:37.634661 (+ 51us) server_negotiation.cc:1035] Waiting for connection context
0809 15:35:37.634990 (+ 329us) server_negotiation.cc:299] Negotiation successful
0809 15:35:37.635062 (+ 72us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":38}
I20260809 15:35:37.635187 4567 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:35:37.630006 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.89.194:43467 (local address 127.3.89.195:33345)
0809 15:35:37.630145 (+ 139us) server_negotiation.cc:206] Beginning negotiation
0809 15:35:37.630150 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 15:35:37.630350 (+ 200us) server_negotiation.cc:407] Connection header received
0809 15:35:37.630511 (+ 161us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 15:35:37.630514 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 15:35:37.630568 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 15:35:37.630649 (+ 81us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 15:35:37.631119 (+ 470us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:35:37.631904 (+ 785us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:35:37.633883 (+ 1979us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:35:37.634486 (+ 603us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:35:37.634610 (+ 124us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:35:37.634661 (+ 51us) server_negotiation.cc:1035] Waiting for connection context
0809 15:35:37.634995 (+ 334us) server_negotiation.cc:299] Negotiation successful
0809 15:35:37.635062 (+ 67us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":59}
I20260809 15:35:37.635213 4583 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:35:37.629962 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.89.193:39293 (local address 127.3.89.194:55555)
0809 15:35:37.630304 (+ 342us) negotiation.cc:107] Waiting for socket to connect
0809 15:35:37.630318 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0809 15:35:37.630403 (+ 85us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 15:35:37.630700 (+ 297us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 15:35:37.630703 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 15:35:37.630765 (+ 62us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 15:35:37.631004 (+ 239us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:35:37.631010 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:35:37.632039 (+ 1029us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:35:37.632043 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:35:37.633749 (+ 1706us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:35:37.633761 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:35:37.634614 (+ 853us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:35:37.634618 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:35:37.634847 (+ 229us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:35:37.634876 (+ 29us) client_negotiation.cc:769] Sending connection context
0809 15:35:37.634946 (+ 70us) client_negotiation.cc:240] Negotiation successful
0809 15:35:37.635015 (+ 69us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":277,"mutex_wait_us":272,"spinlock_wait_cycles":17920,"thread_start_us":69,"threads_started":1}
I20260809 15:35:37.635227 4582 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:35:37.629872 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.89.195:33345 (local address 127.3.89.194:43467)
0809 15:35:37.630244 (+ 372us) negotiation.cc:107] Waiting for socket to connect
0809 15:35:37.630262 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0809 15:35:37.630391 (+ 129us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 15:35:37.630700 (+ 309us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 15:35:37.630703 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 15:35:37.630765 (+ 62us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 15:35:37.630997 (+ 232us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:35:37.631006 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:35:37.632039 (+ 1033us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:35:37.632043 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:35:37.633749 (+ 1706us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:35:37.633761 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:35:37.634614 (+ 853us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:35:37.634618 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:35:37.634781 (+ 163us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:35:37.634832 (+ 51us) client_negotiation.cc:769] Sending connection context
0809 15:35:37.634910 (+ 78us) client_negotiation.cc:240] Negotiation successful
0809 15:35:37.635012 (+ 102us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":301,"mutex_wait_us":279,"spinlock_wait_cycles":48384,"thread_start_us":96,"threads_started":1}
I20260809 15:35:37.635995 4212 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b88f8fc9048e421eab27392b7d0d4322" candidate_uuid: "f65c01d090d9479fb7afeda5bdf5539b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1c5a51a59f73414aa719a4afede261a0" is_pre_election: true
I20260809 15:35:37.636490 4212 raft_consensus.cc:2454] T b88f8fc9048e421eab27392b7d0d4322 P 1c5a51a59f73414aa719a4afede261a0 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f65c01d090d9479fb7afeda5bdf5539b in term 0.
I20260809 15:35:37.636448 4504 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b88f8fc9048e421eab27392b7d0d4322" candidate_uuid: "f65c01d090d9479fb7afeda5bdf5539b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b8ccc17f0dd246cfb5f23f58781a2600" is_pre_election: true
I20260809 15:35:37.636817 4504 raft_consensus.cc:2454] T b88f8fc9048e421eab27392b7d0d4322 P b8ccc17f0dd246cfb5f23f58781a2600 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f65c01d090d9479fb7afeda5bdf5539b in term 0.
I20260809 15:35:37.637118 4284 leader_election.cc:302] T b88f8fc9048e421eab27392b7d0d4322 P f65c01d090d9479fb7afeda5bdf5539b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1c5a51a59f73414aa719a4afede261a0, f65c01d090d9479fb7afeda5bdf5539b; no voters:
I20260809 15:35:37.637423 4577 raft_consensus.cc:2781] T b88f8fc9048e421eab27392b7d0d4322 P f65c01d090d9479fb7afeda5bdf5539b [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260809 15:35:37.637506 4577 raft_consensus.cc:482] T b88f8fc9048e421eab27392b7d0d4322 P f65c01d090d9479fb7afeda5bdf5539b [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 15:35:37.637558 4577 raft_consensus.cc:3037] T b88f8fc9048e421eab27392b7d0d4322 P f65c01d090d9479fb7afeda5bdf5539b [term 0 FOLLOWER]: Advancing to term 1
I20260809 15:35:37.638859 4577 raft_consensus.cc:504] T b88f8fc9048e421eab27392b7d0d4322 P f65c01d090d9479fb7afeda5bdf5539b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8ccc17f0dd246cfb5f23f58781a2600" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 33345 } } peers { permanent_uuid: "f65c01d090d9479fb7afeda5bdf5539b" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 37725 } } peers { permanent_uuid: "1c5a51a59f73414aa719a4afede261a0" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 39293 } }
I20260809 15:35:37.639273 4577 leader_election.cc:288] T b88f8fc9048e421eab27392b7d0d4322 P f65c01d090d9479fb7afeda5bdf5539b [CANDIDATE]: Term 1 election: Requested vote from peers b8ccc17f0dd246cfb5f23f58781a2600 (127.3.89.195:33345), 1c5a51a59f73414aa719a4afede261a0 (127.3.89.193:39293)
I20260809 15:35:37.639667 4504 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b88f8fc9048e421eab27392b7d0d4322" candidate_uuid: "f65c01d090d9479fb7afeda5bdf5539b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b8ccc17f0dd246cfb5f23f58781a2600"
I20260809 15:35:37.639797 4504 raft_consensus.cc:3037] T b88f8fc9048e421eab27392b7d0d4322 P b8ccc17f0dd246cfb5f23f58781a2600 [term 0 FOLLOWER]: Advancing to term 1
I20260809 15:35:37.639832 4212 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b88f8fc9048e421eab27392b7d0d4322" candidate_uuid: "f65c01d090d9479fb7afeda5bdf5539b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1c5a51a59f73414aa719a4afede261a0"
I20260809 15:35:37.639988 4212 raft_consensus.cc:3037] T b88f8fc9048e421eab27392b7d0d4322 P 1c5a51a59f73414aa719a4afede261a0 [term 0 FOLLOWER]: Advancing to term 1
I20260809 15:35:37.640869 4504 raft_consensus.cc:2454] T b88f8fc9048e421eab27392b7d0d4322 P b8ccc17f0dd246cfb5f23f58781a2600 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f65c01d090d9479fb7afeda5bdf5539b in term 1.
I20260809 15:35:37.641150 4212 raft_consensus.cc:2454] T b88f8fc9048e421eab27392b7d0d4322 P 1c5a51a59f73414aa719a4afede261a0 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f65c01d090d9479fb7afeda5bdf5539b in term 1.
I20260809 15:35:37.641202 4287 leader_election.cc:302] T b88f8fc9048e421eab27392b7d0d4322 P f65c01d090d9479fb7afeda5bdf5539b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b8ccc17f0dd246cfb5f23f58781a2600, f65c01d090d9479fb7afeda5bdf5539b; no voters:
I20260809 15:35:37.641367 4577 raft_consensus.cc:2781] T b88f8fc9048e421eab27392b7d0d4322 P f65c01d090d9479fb7afeda5bdf5539b [term 1 FOLLOWER]: Leader election won for term 1
I20260809 15:35:37.641628 4577 raft_consensus.cc:686] T b88f8fc9048e421eab27392b7d0d4322 P f65c01d090d9479fb7afeda5bdf5539b [term 1 LEADER]: Becoming Leader. State: Replica: f65c01d090d9479fb7afeda5bdf5539b, State: Running, Role: LEADER
I20260809 15:35:37.641963 4577 consensus_queue.cc:237] T b88f8fc9048e421eab27392b7d0d4322 P f65c01d090d9479fb7afeda5bdf5539b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8ccc17f0dd246cfb5f23f58781a2600" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 33345 } } peers { permanent_uuid: "f65c01d090d9479fb7afeda5bdf5539b" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 37725 } } peers { permanent_uuid: "1c5a51a59f73414aa719a4afede261a0" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 39293 } }
I20260809 15:35:37.645191 4051 catalog_manager.cc:5495] T b88f8fc9048e421eab27392b7d0d4322 P f65c01d090d9479fb7afeda5bdf5539b reported cstate change: term changed from 0 to 1, leader changed from <none> to f65c01d090d9479fb7afeda5bdf5539b (127.3.89.194). New cstate: current_term: 1 leader_uuid: "f65c01d090d9479fb7afeda5bdf5539b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b8ccc17f0dd246cfb5f23f58781a2600" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 33345 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f65c01d090d9479fb7afeda5bdf5539b" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 37725 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "1c5a51a59f73414aa719a4afede261a0" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 39293 } health_report { overall_health: UNKNOWN } } }
Aug 09 15:35:37 dist-test-slave-2w3w krb5kdc[4003](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786289736, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.3.89.193@KRBTEST.COM
I20260809 15:35:37.690026 4569 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:35:37.676619 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34742 (local address 127.3.89.193:39293)
0809 15:35:37.676792 (+ 173us) server_negotiation.cc:206] Beginning negotiation
0809 15:35:37.676798 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 15:35:37.676984 (+ 186us) server_negotiation.cc:407] Connection header received
0809 15:35:37.677140 (+ 156us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 15:35:37.677143 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 15:35:37.677192 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 15:35:37.677278 (+ 86us) server_negotiation.cc:226] Negotiated authn=SASL
0809 15:35:37.677995 (+ 717us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:35:37.678504 (+ 509us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:35:37.679380 (+ 876us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:35:37.679600 (+ 220us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:35:37.679704 (+ 104us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:35:37.686305 (+ 6601us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 15:35:37.686320 (+ 15us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 15:35:37.686337 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 15:35:37.686362 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0809 15:35:37.688159 (+ 1797us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 15:35:37.688652 (+ 493us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 15:35:37.688657 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 15:35:37.688658 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 15:35:37.688773 (+ 115us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 15:35:37.689085 (+ 312us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 15:35:37.689088 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 15:35:37.689089 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 15:35:37.689343 (+ 254us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 15:35:37.689502 (+ 159us) server_negotiation.cc:1035] Waiting for connection context
0809 15:35:37.689720 (+ 218us) server_negotiation.cc:299] Negotiation successful
0809 15:35:37.689828 (+ 108us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":71}
Aug 09 15:35:37 dist-test-slave-2w3w krb5kdc[4003](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786289737, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Aug 09 15:35:37 dist-test-slave-2w3w krb5kdc[4003](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786289737, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.3.89.193@KRBTEST.COM
I20260809 15:35:37.718801 4569 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:35:37.710058 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34748 (local address 127.3.89.193:39293)
0809 15:35:37.710226 (+ 168us) server_negotiation.cc:206] Beginning negotiation
0809 15:35:37.710229 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0809 15:35:37.710277 (+ 48us) server_negotiation.cc:407] Connection header received
0809 15:35:37.710426 (+ 149us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 15:35:37.710428 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 15:35:37.710473 (+ 45us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 15:35:37.710538 (+ 65us) server_negotiation.cc:226] Negotiated authn=SASL
0809 15:35:37.711325 (+ 787us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:35:37.711896 (+ 571us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:35:37.712930 (+ 1034us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:35:37.713153 (+ 223us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:35:37.713232 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:35:37.715784 (+ 2552us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 15:35:37.715787 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 15:35:37.715790 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 15:35:37.715813 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0809 15:35:37.717219 (+ 1406us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 15:35:37.717632 (+ 413us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 15:35:37.717635 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 15:35:37.717636 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 15:35:37.717709 (+ 73us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 15:35:37.717943 (+ 234us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 15:35:37.717945 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 15:35:37.717946 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 15:35:37.718103 (+ 157us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 15:35:37.718197 (+ 94us) server_negotiation.cc:1035] Waiting for connection context
0809 15:35:37.718455 (+ 258us) server_negotiation.cc:299] Negotiation successful
0809 15:35:37.718555 (+ 100us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":70}
Aug 09 15:35:37 dist-test-slave-2w3w krb5kdc[4003](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786289737, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Aug 09 15:35:37 dist-test-slave-2w3w krb5kdc[4003](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786289737, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.3.89.193@KRBTEST.COM
I20260809 15:35:37.744908 4569 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:35:37.736771 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34756 (local address 127.3.89.193:39293)
0809 15:35:37.736933 (+ 162us) server_negotiation.cc:206] Beginning negotiation
0809 15:35:37.736939 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 15:35:37.737079 (+ 140us) server_negotiation.cc:407] Connection header received
0809 15:35:37.737246 (+ 167us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 15:35:37.737248 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 15:35:37.737300 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 15:35:37.737382 (+ 82us) server_negotiation.cc:226] Negotiated authn=SASL
0809 15:35:37.738064 (+ 682us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:35:37.738563 (+ 499us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:35:37.739615 (+ 1052us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:35:37.739840 (+ 225us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:35:37.739987 (+ 147us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:35:37.742327 (+ 2340us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 15:35:37.742330 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 15:35:37.742332 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 15:35:37.742353 (+ 21us) server_negotiation.cc:922] calling sasl_server_start()
0809 15:35:37.743594 (+ 1241us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 15:35:37.743968 (+ 374us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 15:35:37.743971 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 15:35:37.743972 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 15:35:37.744041 (+ 69us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 15:35:37.744261 (+ 220us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 15:35:37.744263 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 15:35:37.744264 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 15:35:37.744416 (+ 152us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 15:35:37.744509 (+ 93us) server_negotiation.cc:1035] Waiting for connection context
0809 15:35:37.744700 (+ 191us) server_negotiation.cc:299] Negotiation successful
0809 15:35:37.744789 (+ 89us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":54}
W20260809 15:35:37.746433 4166 tablet_service.cc:2908] Rejecting scan request for tablet b88f8fc9048e421eab27392b7d0d4322: Uninitialized: clean time has not yet been initialized
I20260809 15:35:37.748845 3431 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskm314YR/build/debug/bin/kudu with pid 4120
I20260809 15:35:37.756347 3431 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskm314YR/build/debug/bin/kudu with pid 4267
I20260809 15:35:37.763247 3431 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskm314YR/build/debug/bin/kudu with pid 4413
I20260809 15:35:37.769325 3431 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskm314YR/build/debug/bin/kudu with pid 4017
2026-08-09T15:35:37Z chronyd exiting
[ OK ] SecurityITest.TestAuthorizationOnChecksum (2594 ms)
[ RUN ] SecurityITest.SmokeTestAsAuthorizedUser
Loading random data
Initializing database '/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786289732823074-3431-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 15:35:37 dist-test-slave-2w3w krb5kdc[4599](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 15:35:37 dist-test-slave-2w3w krb5kdc[4599](info): set up 2 sockets
Aug 09 15:35:37 dist-test-slave-2w3w krb5kdc[4599](info): commencing operation
krb5kdc: starting...
W20260809 15:35:38.851758 3431 mini_kdc.cc:121] Time spent starting KDC: real 1.059s user 0.004s sys 0.003s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786289732823074-3431-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786289732823074-3431-0/krb5kdc/test-user.keytab.
Aug 09 15:35:38 dist-test-slave-2w3w krb5kdc[4599](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786289738, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T15:35:38Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T15:35:38Z Disabled control of system clock
WARNING: no policy specified for kudu/127.3.89.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.3.89.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.3.89.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786289732823074-3431-0/krb5kdc/kudu_127.3.89.254.keytab.
Entry for principal kudu/127.3.89.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786289732823074-3431-0/krb5kdc/kudu_127.3.89.254.keytab.
I20260809 15:35:38.978516 3431 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskm314YR/build/debug/bin/kudu
/tmp/dist-test-taskm314YR/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786289732823074-3431-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786289732823074-3431-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786289732823074-3431-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786289732823074-3431-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.3.89.254:45563
--webserver_interface=127.3.89.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786289732823074-3431-0/krb5kdc/kudu_127.3.89.254.keytab
--principal=kudu/127.3.89.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.3.89.212:43255
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.3.89.254:45563
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--txn_manager_enabled=true with env {KRB5CCNAME=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786289732823074-3431-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786289732823074-3431-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786289732823074-3431-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 15:35:39.090610 4616 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 15:35:39.090888 4616 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 15:35:39.091004 4616 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 15:35:39.095111 4616 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 15:35:39.095184 4616 flags.cc:425] Enabled experimental flag: --txn_manager_enabled=true
W20260809 15:35:39.095213 4616 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 15:35:39.095258 4616 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 15:35:39.095296 4616 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 15:35:39.095328 4616 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 15:35:39.100524 4616 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.3.89.212:43255
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786289732823074-3431-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786289732823074-3431-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.3.89.254:45563
--txn_manager_enabled=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786289732823074-3431-0/krb5kdc/kudu_127.3.89.254.keytab
--principal=kudu/127.3.89.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.3.89.254:45563
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786289732823074-3431-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.3.89.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.4616
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786289732823074-3431-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 09 Aug 2026 14:46:55 UTC on ccycloud.quasar-yswkmf.root.comops.site
build id 4576540
I20260809 15:35:39.101522 4616 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 15:35:39.102519 4616 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 15:35:39.110809 4622 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 15:35:39.110846 4621 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 15:35:39.110810 4624 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 15:35:39.111016 4616 server_base.cc:1034] running on GCE node
I20260809 15:35:39.111660 4616 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 15:35:39.112742 4616 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 15:35:39.113937 4616 hybrid_clock.cc:648] HybridClock initialized: now 1786289739113913 us; error 35 us; skew 500 ppm
Aug 09 15:35:39 dist-test-slave-2w3w krb5kdc[4599](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786289739, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.89.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 15:35:39.116618 4616 init.cc:377] Logged in from keytab as kudu/127.3.89.254@KRBTEST.COM (short username kudu)
I20260809 15:35:39.117443 4616 webserver.cc:460] Webserver started at http://127.3.89.254:34221/ using document root <none> and password file <none>
I20260809 15:35:39.117970 4616 fs_manager.cc:359] Metadata directory not provided
I20260809 15:35:39.118057 4616 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 15:35:39.118273 4616 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 15:35:39.120278 4616 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786289732823074-3431-0/minicluster-data/master-0/data/instance:
uuid: "2d52ec43c6d145ccbb52cd484ca05989"
format_stamp: "Formatted at 2026-08-09 15:35:39 on dist-test-slave-2w3w"
server_key: "9b82beac4f7f90316972011411ebcf4f"
server_key_iv: "78f60f4aed116663e0cab9b521ddb34e"
server_key_version: "encryptionkey@0"
I20260809 15:35:39.120838 4616 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786289732823074-3431-0/minicluster-data/master-0/wal/instance:
uuid: "2d52ec43c6d145ccbb52cd484ca05989"
format_stamp: "Formatted at 2026-08-09 15:35:39 on dist-test-slave-2w3w"
server_key: "9b82beac4f7f90316972011411ebcf4f"
server_key_iv: "78f60f4aed116663e0cab9b521ddb34e"
server_key_version: "encryptionkey@0"
I20260809 15:35:39.124352 4616 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.004s sys 0.000s
I20260809 15:35:39.126523 4631 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 15:35:39.127693 4616 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260809 15:35:39.127849 4616 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786289732823074-3431-0/minicluster-data/master-0/data,/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786289732823074-3431-0/minicluster-data/master-0/wal
uuid: "2d52ec43c6d145ccbb52cd484ca05989"
format_stamp: "Formatted at 2026-08-09 15:35:39 on dist-test-slave-2w3w"
server_key: "9b82beac4f7f90316972011411ebcf4f"
server_key_iv: "78f60f4aed116663e0cab9b521ddb34e"
server_key_version: "encryptionkey@0"
I20260809 15:35:39.127959 4616 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786289732823074-3431-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786289732823074-3431-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786289732823074-3431-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 15:35:39.150548 4616 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 15:35:39.153282 4616 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 15:35:39.153474 4616 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 15:35:39.161540 4616 rpc_server.cc:307] RPC server started. Bound to: 127.3.89.254:45563
I20260809 15:35:39.161571 4697 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.3.89.254:45563 every 8 connection(s)
I20260809 15:35:39.162420 4616 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786289732823074-3431-0/minicluster-data/master-0/data/info.pb
I20260809 15:35:39.164490 3431 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskm314YR/build/debug/bin/kudu as pid 4616
I20260809 15:35:39.164585 3431 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786289732823074-3431-0/minicluster-data/master-0/wal/instance
I20260809 15:35:39.164858 3431 external_mini_cluster.cc:1428] Setting key b1a894866555ba1b43582b3e3bc1e565
I20260809 15:35:39.165108 4698 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 15:35:39.171169 4698 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2d52ec43c6d145ccbb52cd484ca05989: Bootstrap starting.
I20260809 15:35:39.173508 4698 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 2d52ec43c6d145ccbb52cd484ca05989: Neither blocks nor log segments found. Creating new log.
Aug 09 15:35:39 dist-test-slave-2w3w krb5kdc[4599](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786289738, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.3.89.254@KRBTEST.COM
I20260809 15:35:39.174404 4698 log.cc:824] T 00000000000000000000000000000000 P 2d52ec43c6d145ccbb52cd484ca05989: Log is configured to *not* fsync() on all Append() calls
I20260809 15:35:39.176436 4698 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2d52ec43c6d145ccbb52cd484ca05989: No bootstrap required, opened a new log
I20260809 15:35:39.179870 4701 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:35:39.166206 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46744 (local address 127.3.89.254:45563)
0809 15:35:39.166630 (+ 424us) server_negotiation.cc:206] Beginning negotiation
0809 15:35:39.166645 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0809 15:35:39.166680 (+ 35us) server_negotiation.cc:407] Connection header received
0809 15:35:39.167703 (+ 1023us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 15:35:39.167721 (+ 18us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 15:35:39.168025 (+ 304us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 15:35:39.168376 (+ 351us) server_negotiation.cc:226] Negotiated authn=SASL
0809 15:35:39.169146 (+ 770us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:35:39.170141 (+ 995us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:35:39.171003 (+ 862us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:35:39.171529 (+ 526us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:35:39.171699 (+ 170us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:35:39.174447 (+ 2748us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 15:35:39.174452 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 15:35:39.174463 (+ 11us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 15:35:39.174515 (+ 52us) server_negotiation.cc:922] calling sasl_server_start()
0809 15:35:39.177011 (+ 2496us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 15:35:39.177513 (+ 502us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 15:35:39.177517 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 15:35:39.177521 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 15:35:39.177632 (+ 111us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 15:35:39.177921 (+ 289us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 15:35:39.177923 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 15:35:39.177925 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 15:35:39.178261 (+ 336us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 15:35:39.178408 (+ 147us) server_negotiation.cc:1035] Waiting for connection context
0809 15:35:39.178868 (+ 460us) server_negotiation.cc:299] Negotiation successful
0809 15:35:39.179169 (+ 301us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":295,"thread_start_us":154,"threads_started":1}
I20260809 15:35:39.179880 4698 raft_consensus.cc:348] T 00000000000000000000000000000000 P 2d52ec43c6d145ccbb52cd484ca05989 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2d52ec43c6d145ccbb52cd484ca05989" member_type: VOTER last_known_addr { host: "127.3.89.254" port: 45563 } }
I20260809 15:35:39.180210 4698 raft_consensus.cc:374] T 00000000000000000000000000000000 P 2d52ec43c6d145ccbb52cd484ca05989 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 15:35:39.180284 4698 raft_consensus.cc:729] T 00000000000000000000000000000000 P 2d52ec43c6d145ccbb52cd484ca05989 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2d52ec43c6d145ccbb52cd484ca05989, State: Initialized, Role: FOLLOWER
I20260809 15:35:39.180677 4698 consensus_queue.cc:260] T 00000000000000000000000000000000 P 2d52ec43c6d145ccbb52cd484ca05989 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2d52ec43c6d145ccbb52cd484ca05989" member_type: VOTER last_known_addr { host: "127.3.89.254" port: 45563 } }
I20260809 15:35:39.180830 4698 raft_consensus.cc:388] T 00000000000000000000000000000000 P 2d52ec43c6d145ccbb52cd484ca05989 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 15:35:39.180928 4698 raft_consensus.cc:482] T 00000000000000000000000000000000 P 2d52ec43c6d145ccbb52cd484ca05989 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 15:35:39.181037 4698 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 2d52ec43c6d145ccbb52cd484ca05989 [term 0 FOLLOWER]: Advancing to term 1
I20260809 15:35:39.182046 4698 raft_consensus.cc:504] T 00000000000000000000000000000000 P 2d52ec43c6d145ccbb52cd484ca05989 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2d52ec43c6d145ccbb52cd484ca05989" member_type: VOTER last_known_addr { host: "127.3.89.254" port: 45563 } }
I20260809 15:35:39.182367 4698 leader_election.cc:302] T 00000000000000000000000000000000 P 2d52ec43c6d145ccbb52cd484ca05989 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 2d52ec43c6d145ccbb52cd484ca05989; no voters:
I20260809 15:35:39.182643 4698 leader_election.cc:288] T 00000000000000000000000000000000 P 2d52ec43c6d145ccbb52cd484ca05989 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 15:35:39.182780 4703 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 2d52ec43c6d145ccbb52cd484ca05989 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 15:35:39.183043 4703 raft_consensus.cc:686] T 00000000000000000000000000000000 P 2d52ec43c6d145ccbb52cd484ca05989 [term 1 LEADER]: Becoming Leader. State: Replica: 2d52ec43c6d145ccbb52cd484ca05989, State: Running, Role: LEADER
I20260809 15:35:39.183451 4703 consensus_queue.cc:237] T 00000000000000000000000000000000 P 2d52ec43c6d145ccbb52cd484ca05989 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2d52ec43c6d145ccbb52cd484ca05989" member_type: VOTER last_known_addr { host: "127.3.89.254" port: 45563 } }
I20260809 15:35:39.183794 4698 sys_catalog.cc:564] T 00000000000000000000000000000000 P 2d52ec43c6d145ccbb52cd484ca05989 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 15:35:39.185086 4704 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2d52ec43c6d145ccbb52cd484ca05989 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "2d52ec43c6d145ccbb52cd484ca05989" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2d52ec43c6d145ccbb52cd484ca05989" member_type: VOTER last_known_addr { host: "127.3.89.254" port: 45563 } } }
I20260809 15:35:39.185220 4704 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2d52ec43c6d145ccbb52cd484ca05989 [sys.catalog]: This master's current role is: LEADER
I20260809 15:35:39.185647 4714 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 15:35:39.186331 4705 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2d52ec43c6d145ccbb52cd484ca05989 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 2d52ec43c6d145ccbb52cd484ca05989. Latest consensus state: current_term: 1 leader_uuid: "2d52ec43c6d145ccbb52cd484ca05989" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2d52ec43c6d145ccbb52cd484ca05989" member_type: VOTER last_known_addr { host: "127.3.89.254" port: 45563 } } }
I20260809 15:35:39.186532 4705 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2d52ec43c6d145ccbb52cd484ca05989 [sys.catalog]: This master's current role is: LEADER
I20260809 15:35:39.188484 4714 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 15:35:39.193707 4714 catalog_manager.cc:1347] Generated new cluster ID: 08fed2e1f6ad47ba9a99993433d24468
I20260809 15:35:39.193769 4714 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 15:35:39.209537 4714 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 15:35:39.210367 4714 catalog_manager.cc:1504] Loading token signing keys...
I20260809 15:35:39.215092 4714 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 2d52ec43c6d145ccbb52cd484ca05989: Generated new TSK 0
I20260809 15:35:39.215670 4714 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.3.89.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.3.89.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.3.89.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786289732823074-3431-0/krb5kdc/kudu_127.3.89.193.keytab.
Entry for principal kudu/127.3.89.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786289732823074-3431-0/krb5kdc/kudu_127.3.89.193.keytab.
I20260809 15:35:39.249980 3431 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskm314YR/build/debug/bin/kudu
/tmp/dist-test-taskm314YR/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786289732823074-3431-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786289732823074-3431-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786289732823074-3431-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786289732823074-3431-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.3.89.193:0
--local_ip_for_outbound_sockets=127.3.89.193
--webserver_interface=127.3.89.193
--webserver_port=0
--tserver_master_addrs=127.3.89.254:45563
--keytab_file=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786289732823074-3431-0/krb5kdc/kudu_127.3.89.193.keytab
--principal=kudu/127.3.89.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.3.89.212:43255
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786289732823074-3431-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786289732823074-3431-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786289732823074-3431-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 15:35:39.362483 4725 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 15:35:39.362735 4725 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 15:35:39.362779 4725 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 15:35:39.366977 4725 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 15:35:39.367043 4725 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 15:35:39.367082 4725 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260809 15:35:39.367120 4725 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.3.89.193
I20260809 15:35:39.372280 4725 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.3.89.212:43255
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786289732823074-3431-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786289732823074-3431-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786289732823074-3431-0/krb5kdc/kudu_127.3.89.193.keytab
--principal=kudu/127.3.89.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.3.89.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786289732823074-3431-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.3.89.193
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.3.89.254:45563
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.4725
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.3.89.193
--log_dir=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786289732823074-3431-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 09 Aug 2026 14:46:55 UTC on ccycloud.quasar-yswkmf.root.comops.site
build id 4576540
I20260809 15:35:39.373270 4725 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 15:35:39.374224 4725 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 15:35:39.383934 4733 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 15:35:39.383951 4731 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 15:35:39.383957 4730 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 15:35:39.384294 4725 server_base.cc:1034] running on GCE node
I20260809 15:35:39.384742 4725 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 15:35:39.385367 4725 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 15:35:39.386559 4725 hybrid_clock.cc:648] HybridClock initialized: now 1786289739386537 us; error 55 us; skew 500 ppm
Aug 09 15:35:39 dist-test-slave-2w3w krb5kdc[4599](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786289739, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.89.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 15:35:39.389578 4725 init.cc:377] Logged in from keytab as kudu/127.3.89.193@KRBTEST.COM (short username kudu)
I20260809 15:35:39.390584 4725 webserver.cc:460] Webserver started at http://127.3.89.193:37701/ using document root <none> and password file <none>
I20260809 15:35:39.391263 4725 fs_manager.cc:359] Metadata directory not provided
I20260809 15:35:39.391342 4725 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 15:35:39.391606 4725 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 15:35:39.393924 4725 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786289732823074-3431-0/minicluster-data/ts-0/data/instance:
uuid: "6383b28fd60f46e6b9627721662f1918"
format_stamp: "Formatted at 2026-08-09 15:35:39 on dist-test-slave-2w3w"
server_key: "ff73a86ffa546fd9e6cecd641b5ad118"
server_key_iv: "a2f029b825f38577ba7cba0013c001fc"
server_key_version: "encryptionkey@0"
I20260809 15:35:39.394543 4725 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786289732823074-3431-0/minicluster-data/ts-0/wal/instance:
uuid: "6383b28fd60f46e6b9627721662f1918"
format_stamp: "Formatted at 2026-08-09 15:35:39 on dist-test-slave-2w3w"
server_key: "ff73a86ffa546fd9e6cecd641b5ad118"
server_key_iv: "a2f029b825f38577ba7cba0013c001fc"
server_key_version: "encryptionkey@0"
I20260809 15:35:39.398881 4725 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.007s sys 0.000s
I20260809 15:35:39.401512 4740 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 15:35:39.402719 4725 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.000s sys 0.002s
I20260809 15:35:39.402886 4725 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786289732823074-3431-0/minicluster-data/ts-0/data,/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786289732823074-3431-0/minicluster-data/ts-0/wal
uuid: "6383b28fd60f46e6b9627721662f1918"
format_stamp: "Formatted at 2026-08-09 15:35:39 on dist-test-slave-2w3w"
server_key: "ff73a86ffa546fd9e6cecd641b5ad118"
server_key_iv: "a2f029b825f38577ba7cba0013c001fc"
server_key_version: "encryptionkey@0"
I20260809 15:35:39.403057 4725 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786289732823074-3431-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786289732823074-3431-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786289732823074-3431-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 15:35:39.423136 4725 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 15:35:39.426128 4725 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 15:35:39.426327 4725 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 15:35:39.427956 4725 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 15:35:39.428030 4725 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 15:35:39.428105 4725 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 15:35:39.428158 4725 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 09 15:35:39 dist-test-slave-2w3w krb5kdc[4599](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786289739, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.89.193@KRBTEST.COM for kudu/127.3.89.254@KRBTEST.COM
I20260809 15:35:39.438937 4725 rpc_server.cc:307] RPC server started. Bound to: 127.3.89.193:44379
I20260809 15:35:39.439065 4866 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.3.89.193:44379 every 8 connection(s)
I20260809 15:35:39.439911 4725 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786289732823074-3431-0/minicluster-data/ts-0/data/info.pb
I20260809 15:35:39.442204 4701 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:35:39.428635 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.89.193:55949 (local address 127.3.89.254:45563)
0809 15:35:39.428806 (+ 171us) server_negotiation.cc:206] Beginning negotiation
0809 15:35:39.428812 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 15:35:39.430158 (+ 1346us) server_negotiation.cc:407] Connection header received
0809 15:35:39.431712 (+ 1554us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 15:35:39.431717 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 15:35:39.431784 (+ 67us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 15:35:39.431901 (+ 117us) server_negotiation.cc:226] Negotiated authn=SASL
0809 15:35:39.433194 (+ 1293us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:35:39.433955 (+ 761us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:35:39.435313 (+ 1358us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:35:39.435564 (+ 251us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:35:39.435681 (+ 117us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:35:39.438341 (+ 2660us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 15:35:39.438344 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 15:35:39.438346 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 15:35:39.438371 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0809 15:35:39.439977 (+ 1606us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 15:35:39.440643 (+ 666us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 15:35:39.440649 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 15:35:39.440652 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 15:35:39.440744 (+ 92us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 15:35:39.441105 (+ 361us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 15:35:39.441109 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 15:35:39.441111 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 15:35:39.441306 (+ 195us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 15:35:39.441417 (+ 111us) server_negotiation.cc:1035] Waiting for connection context
0809 15:35:39.441884 (+ 467us) server_negotiation.cc:299] Negotiation successful
0809 15:35:39.442009 (+ 125us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":64}
I20260809 15:35:39.442912 4749 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:35:39.429512 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.89.254:45563 (local address 127.3.89.193:55949)
0809 15:35:39.429993 (+ 481us) negotiation.cc:107] Waiting for socket to connect
0809 15:35:39.430052 (+ 59us) client_negotiation.cc:174] Beginning negotiation
0809 15:35:39.431450 (+ 1398us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 15:35:39.432114 (+ 664us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 15:35:39.432124 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 15:35:39.432477 (+ 353us) client_negotiation.cc:189] Negotiated authn=SASL
0809 15:35:39.433039 (+ 562us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:35:39.433056 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:35:39.434085 (+ 1029us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:35:39.434089 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:35:39.435186 (+ 1097us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:35:39.435196 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:35:39.435719 (+ 523us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:35:39.435723 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:35:39.435899 (+ 176us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:35:39.436478 (+ 579us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 15:35:39.436494 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0809 15:35:39.438176 (+ 1682us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 15:35:39.440116 (+ 1940us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 15:35:39.440121 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 15:35:39.440134 (+ 13us) client_negotiation.cc:760] calling sasl_client_step()
0809 15:35:39.440507 (+ 373us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 15:35:39.440853 (+ 346us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 15:35:39.440858 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 15:35:39.440860 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 15:35:39.441002 (+ 142us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 15:35:39.441417 (+ 415us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 15:35:39.441423 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 15:35:39.441649 (+ 226us) client_negotiation.cc:769] Sending connection context
0809 15:35:39.441840 (+ 191us) client_negotiation.cc:240] Negotiation successful
0809 15:35:39.442142 (+ 302us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":251,"thread_start_us":105,"threads_started":1}
I20260809 15:35:39.444321 4867 heartbeater.cc:347] Connected to a master server at 127.3.89.254:45563
I20260809 15:35:39.444473 4867 heartbeater.cc:464] Registering TS with master...
I20260809 15:35:39.444955 4867 heartbeater.cc:511] Master 127.3.89.254:45563 requested a full tablet report, sending...
I20260809 15:35:39.446424 4649 ts_manager.cc:194] Registered new tserver with Master: 6383b28fd60f46e6b9627721662f1918 (127.3.89.193:44379)
I20260809 15:35:39.446564 3431 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskm314YR/build/debug/bin/kudu as pid 4725
I20260809 15:35:39.446679 3431 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786289732823074-3431-0/minicluster-data/ts-0/wal/instance
I20260809 15:35:39.447005 3431 external_mini_cluster.cc:1428] Setting key d5598245d07e45f3cce4e74e3170fb32
I20260809 15:35:39.447811 4649 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.3.89.193@KRBTEST.COM'} at 127.3.89.193:55949
I20260809 15:35:39.454190 4873 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:35:39.446367 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.89.254:45563 (local address 127.3.89.193:57043)
0809 15:35:39.446720 (+ 353us) negotiation.cc:107] Waiting for socket to connect
0809 15:35:39.446736 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0809 15:35:39.446816 (+ 80us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 15:35:39.447304 (+ 488us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 15:35:39.447307 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 15:35:39.447502 (+ 195us) client_negotiation.cc:189] Negotiated authn=SASL
0809 15:35:39.447621 (+ 119us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:35:39.447627 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:35:39.448492 (+ 865us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:35:39.448498 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:35:39.449111 (+ 613us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:35:39.449119 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:35:39.449786 (+ 667us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:35:39.449789 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:35:39.449893 (+ 104us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:35:39.450355 (+ 462us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 15:35:39.450376 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0809 15:35:39.450869 (+ 493us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 15:35:39.452620 (+ 1751us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 15:35:39.452623 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 15:35:39.452626 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0809 15:35:39.452890 (+ 264us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 15:35:39.453230 (+ 340us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 15:35:39.453232 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 15:35:39.453234 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 15:35:39.453309 (+ 75us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 15:35:39.453753 (+ 444us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 15:35:39.453755 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 15:35:39.453828 (+ 73us) client_negotiation.cc:769] Sending connection context
0809 15:35:39.453930 (+ 102us) client_negotiation.cc:240] Negotiation successful
0809 15:35:39.454052 (+ 122us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":265,"thread_start_us":115,"threads_started":1}
I20260809 15:35:39.454362 4701 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:35:39.446419 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.89.193:57043 (local address 127.3.89.254:45563)
0809 15:35:39.447098 (+ 679us) server_negotiation.cc:206] Beginning negotiation
0809 15:35:39.447102 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 15:35:39.447115 (+ 13us) server_negotiation.cc:407] Connection header received
0809 15:35:39.447147 (+ 32us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 15:35:39.447149 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 15:35:39.447193 (+ 44us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 15:35:39.447287 (+ 94us) server_negotiation.cc:226] Negotiated authn=SASL
0809 15:35:39.447737 (+ 450us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:35:39.448376 (+ 639us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:35:39.449276 (+ 900us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:35:39.449565 (+ 289us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:35:39.449641 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:35:39.451080 (+ 1439us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 15:35:39.451083 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 15:35:39.451085 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 15:35:39.451106 (+ 21us) server_negotiation.cc:922] calling sasl_server_start()
0809 15:35:39.452489 (+ 1383us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 15:35:39.453034 (+ 545us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 15:35:39.453040 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 15:35:39.453044 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 15:35:39.453135 (+ 91us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 15:35:39.453401 (+ 266us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 15:35:39.453407 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 15:35:39.453412 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0809 15:35:39.453655 (+ 243us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 15:35:39.453758 (+ 103us) server_negotiation.cc:1035] Waiting for connection context
0809 15:35:39.453995 (+ 237us) server_negotiation.cc:299] Negotiation successful
0809 15:35:39.454131 (+ 136us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":28,"server-negotiator.queue_time_us":579}
WARNING: no policy specified for kudu/127.3.89.194@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.3.89.194@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.3.89.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786289732823074-3431-0/krb5kdc/kudu_127.3.89.194.keytab.
Entry for principal kudu/127.3.89.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786289732823074-3431-0/krb5kdc/kudu_127.3.89.194.keytab.
I20260809 15:35:39.477670 3431 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskm314YR/build/debug/bin/kudu
/tmp/dist-test-taskm314YR/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786289732823074-3431-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786289732823074-3431-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786289732823074-3431-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786289732823074-3431-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.3.89.194:0
--local_ip_for_outbound_sockets=127.3.89.194
--webserver_interface=127.3.89.194
--webserver_port=0
--tserver_master_addrs=127.3.89.254:45563
--keytab_file=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786289732823074-3431-0/krb5kdc/kudu_127.3.89.194.keytab
--principal=kudu/127.3.89.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.3.89.212:43255
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786289732823074-3431-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786289732823074-3431-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786289732823074-3431-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 15:35:39.588793 4876 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 15:35:39.589041 4876 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 15:35:39.589087 4876 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 15:35:39.593029 4876 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 15:35:39.593084 4876 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 15:35:39.593120 4876 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260809 15:35:39.593151 4876 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.3.89.194
I20260809 15:35:39.598377 4876 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.3.89.212:43255
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786289732823074-3431-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786289732823074-3431-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786289732823074-3431-0/krb5kdc/kudu_127.3.89.194.keytab
--principal=kudu/127.3.89.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.3.89.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786289732823074-3431-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.3.89.194
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.3.89.254:45563
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.4876
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.3.89.194
--log_dir=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786289732823074-3431-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 09 Aug 2026 14:46:55 UTC on ccycloud.quasar-yswkmf.root.comops.site
build id 4576540
I20260809 15:35:39.599457 4876 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 15:35:39.600478 4876 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 15:35:39.610095 4881 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 15:35:39.610078 4882 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 15:35:39.610183 4876 server_base.cc:1034] running on GCE node
W20260809 15:35:39.610069 4884 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 15:35:39.610634 4876 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 15:35:39.611287 4876 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 15:35:39.612448 4876 hybrid_clock.cc:648] HybridClock initialized: now 1786289739612422 us; error 43 us; skew 500 ppm
Aug 09 15:35:39 dist-test-slave-2w3w krb5kdc[4599](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786289739, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.89.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 15:35:39.615079 4876 init.cc:377] Logged in from keytab as kudu/127.3.89.194@KRBTEST.COM (short username kudu)
I20260809 15:35:39.615867 4876 webserver.cc:460] Webserver started at http://127.3.89.194:46121/ using document root <none> and password file <none>
I20260809 15:35:39.616367 4876 fs_manager.cc:359] Metadata directory not provided
I20260809 15:35:39.616423 4876 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 15:35:39.616592 4876 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 15:35:39.618516 4876 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786289732823074-3431-0/minicluster-data/ts-1/data/instance:
uuid: "8ee7401f0286413092bba932ce0a7ba2"
format_stamp: "Formatted at 2026-08-09 15:35:39 on dist-test-slave-2w3w"
server_key: "1629f4f51e321fafa6b58a268d6e440e"
server_key_iv: "7b05ffa743d90160dc77e7674a2e2233"
server_key_version: "encryptionkey@0"
I20260809 15:35:39.619105 4876 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786289732823074-3431-0/minicluster-data/ts-1/wal/instance:
uuid: "8ee7401f0286413092bba932ce0a7ba2"
format_stamp: "Formatted at 2026-08-09 15:35:39 on dist-test-slave-2w3w"
server_key: "1629f4f51e321fafa6b58a268d6e440e"
server_key_iv: "7b05ffa743d90160dc77e7674a2e2233"
server_key_version: "encryptionkey@0"
I20260809 15:35:39.622764 4876 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.003s sys 0.000s
I20260809 15:35:39.625113 4893 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 15:35:39.626259 4876 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260809 15:35:39.626375 4876 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786289732823074-3431-0/minicluster-data/ts-1/data,/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786289732823074-3431-0/minicluster-data/ts-1/wal
uuid: "8ee7401f0286413092bba932ce0a7ba2"
format_stamp: "Formatted at 2026-08-09 15:35:39 on dist-test-slave-2w3w"
server_key: "1629f4f51e321fafa6b58a268d6e440e"
server_key_iv: "7b05ffa743d90160dc77e7674a2e2233"
server_key_version: "encryptionkey@0"
I20260809 15:35:39.626485 4876 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786289732823074-3431-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786289732823074-3431-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786289732823074-3431-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 15:35:39.640153 4876 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 15:35:39.643347 4876 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 15:35:39.643561 4876 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 15:35:39.645596 4876 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 15:35:39.645677 4876 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 15:35:39.645735 4876 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 15:35:39.645766 4876 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 09 15:35:39 dist-test-slave-2w3w krb5kdc[4599](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786289739, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.89.194@KRBTEST.COM for kudu/127.3.89.254@KRBTEST.COM
I20260809 15:35:39.656551 4876 rpc_server.cc:307] RPC server started. Bound to: 127.3.89.194:44743
I20260809 15:35:39.656649 5021 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.3.89.194:44743 every 8 connection(s)
I20260809 15:35:39.657729 4876 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786289732823074-3431-0/minicluster-data/ts-1/data/info.pb
I20260809 15:35:39.660216 4701 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:35:39.646269 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.89.194:59011 (local address 127.3.89.254:45563)
0809 15:35:39.646441 (+ 172us) server_negotiation.cc:206] Beginning negotiation
0809 15:35:39.646446 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 15:35:39.647873 (+ 1427us) server_negotiation.cc:407] Connection header received
0809 15:35:39.649088 (+ 1215us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 15:35:39.649091 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 15:35:39.649152 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 15:35:39.649231 (+ 79us) server_negotiation.cc:226] Negotiated authn=SASL
0809 15:35:39.650728 (+ 1497us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:35:39.651635 (+ 907us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:35:39.653378 (+ 1743us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:35:39.653614 (+ 236us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:35:39.653733 (+ 119us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:35:39.656632 (+ 2899us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 15:35:39.656639 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 15:35:39.656643 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 15:35:39.656675 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0809 15:35:39.658318 (+ 1643us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 15:35:39.658864 (+ 546us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 15:35:39.658867 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 15:35:39.658868 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 15:35:39.658952 (+ 84us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 15:35:39.659293 (+ 341us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 15:35:39.659296 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 15:35:39.659297 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 15:35:39.659466 (+ 169us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 15:35:39.659545 (+ 79us) server_negotiation.cc:1035] Waiting for connection context
0809 15:35:39.659998 (+ 453us) server_negotiation.cc:299] Negotiation successful
0809 15:35:39.660099 (+ 101us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":56}
I20260809 15:35:39.660971 4902 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:35:39.647267 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.89.254:45563 (local address 127.3.89.194:59011)
0809 15:35:39.647704 (+ 437us) negotiation.cc:107] Waiting for socket to connect
0809 15:35:39.647751 (+ 47us) client_negotiation.cc:174] Beginning negotiation
0809 15:35:39.648848 (+ 1097us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 15:35:39.649401 (+ 553us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 15:35:39.649409 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 15:35:39.649856 (+ 447us) client_negotiation.cc:189] Negotiated authn=SASL
0809 15:35:39.650576 (+ 720us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:35:39.650596 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:35:39.651767 (+ 1171us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:35:39.651771 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:35:39.653252 (+ 1481us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:35:39.653263 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:35:39.653761 (+ 498us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:35:39.653766 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:35:39.653946 (+ 180us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:35:39.654624 (+ 678us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 15:35:39.654646 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0809 15:35:39.656429 (+ 1783us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 15:35:39.658449 (+ 2020us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 15:35:39.658456 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 15:35:39.658468 (+ 12us) client_negotiation.cc:760] calling sasl_client_step()
0809 15:35:39.658755 (+ 287us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 15:35:39.659092 (+ 337us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 15:35:39.659094 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 15:35:39.659095 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0809 15:35:39.659214 (+ 119us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 15:35:39.659554 (+ 340us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 15:35:39.659559 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 15:35:39.659757 (+ 198us) client_negotiation.cc:769] Sending connection context
0809 15:35:39.659943 (+ 186us) client_negotiation.cc:240] Negotiation successful
0809 15:35:39.660162 (+ 219us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":254,"thread_start_us":145,"threads_started":1}
I20260809 15:35:39.662204 5022 heartbeater.cc:347] Connected to a master server at 127.3.89.254:45563
I20260809 15:35:39.662346 5022 heartbeater.cc:464] Registering TS with master...
I20260809 15:35:39.662806 5022 heartbeater.cc:511] Master 127.3.89.254:45563 requested a full tablet report, sending...
I20260809 15:35:39.663808 4651 ts_manager.cc:194] Registered new tserver with Master: 8ee7401f0286413092bba932ce0a7ba2 (127.3.89.194:44743)
I20260809 15:35:39.664391 3431 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskm314YR/build/debug/bin/kudu as pid 4876
I20260809 15:35:39.664520 3431 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786289732823074-3431-0/minicluster-data/ts-1/wal/instance
I20260809 15:35:39.664583 4651 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.3.89.194@KRBTEST.COM'} at 127.3.89.194:59011
I20260809 15:35:39.664872 3431 external_mini_cluster.cc:1428] Setting key 3c03dedf341835858c9fa00ca7446e24
I20260809 15:35:39.671468 5028 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:35:39.664238 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.89.254:45563 (local address 127.3.89.194:45153)
0809 15:35:39.664583 (+ 345us) negotiation.cc:107] Waiting for socket to connect
0809 15:35:39.664599 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0809 15:35:39.664698 (+ 99us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 15:35:39.665053 (+ 355us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 15:35:39.665055 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 15:35:39.665263 (+ 208us) client_negotiation.cc:189] Negotiated authn=SASL
0809 15:35:39.665416 (+ 153us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:35:39.665423 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:35:39.666293 (+ 870us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:35:39.666296 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:35:39.666866 (+ 570us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:35:39.666874 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:35:39.667351 (+ 477us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:35:39.667354 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:35:39.667446 (+ 92us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:35:39.667929 (+ 483us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 15:35:39.667943 (+ 14us) client_negotiation.cc:656] calling sasl_client_start()
0809 15:35:39.668258 (+ 315us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 15:35:39.669852 (+ 1594us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 15:35:39.669856 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 15:35:39.669859 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0809 15:35:39.670054 (+ 195us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 15:35:39.670349 (+ 295us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 15:35:39.670353 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 15:35:39.670355 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 15:35:39.670443 (+ 88us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 15:35:39.670857 (+ 414us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 15:35:39.670860 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 15:35:39.670997 (+ 137us) client_negotiation.cc:769] Sending connection context
0809 15:35:39.671124 (+ 127us) client_negotiation.cc:240] Negotiation successful
0809 15:35:39.671271 (+ 147us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":261,"mutex_wait_us":4,"thread_start_us":108,"threads_started":1}
I20260809 15:35:39.671458 4701 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:35:39.664306 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.89.194:45153 (local address 127.3.89.254:45563)
0809 15:35:39.664401 (+ 95us) server_negotiation.cc:206] Beginning negotiation
0809 15:35:39.664405 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 15:35:39.664676 (+ 271us) server_negotiation.cc:407] Connection header received
0809 15:35:39.664775 (+ 99us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 15:35:39.664778 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 15:35:39.664824 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 15:35:39.664911 (+ 87us) server_negotiation.cc:226] Negotiated authn=SASL
0809 15:35:39.665530 (+ 619us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:35:39.666162 (+ 632us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:35:39.667001 (+ 839us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:35:39.667231 (+ 230us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:35:39.667347 (+ 116us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:35:39.668402 (+ 1055us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 15:35:39.668405 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 15:35:39.668408 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 15:35:39.668432 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0809 15:35:39.669738 (+ 1306us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 15:35:39.670155 (+ 417us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 15:35:39.670159 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 15:35:39.670161 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 15:35:39.670247 (+ 86us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 15:35:39.670534 (+ 287us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 15:35:39.670539 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 15:35:39.670540 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 15:35:39.670756 (+ 216us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 15:35:39.670869 (+ 113us) server_negotiation.cc:1035] Waiting for connection context
0809 15:35:39.671177 (+ 308us) server_negotiation.cc:299] Negotiation successful
0809 15:35:39.671297 (+ 120us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":39}
WARNING: no policy specified for kudu/127.3.89.195@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.3.89.195@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.3.89.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786289732823074-3431-0/krb5kdc/kudu_127.3.89.195.keytab.
Entry for principal kudu/127.3.89.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786289732823074-3431-0/krb5kdc/kudu_127.3.89.195.keytab.
I20260809 15:35:39.694264 3431 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskm314YR/build/debug/bin/kudu
/tmp/dist-test-taskm314YR/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786289732823074-3431-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786289732823074-3431-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786289732823074-3431-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786289732823074-3431-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.3.89.195:0
--local_ip_for_outbound_sockets=127.3.89.195
--webserver_interface=127.3.89.195
--webserver_port=0
--tserver_master_addrs=127.3.89.254:45563
--keytab_file=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786289732823074-3431-0/krb5kdc/kudu_127.3.89.195.keytab
--principal=kudu/127.3.89.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.3.89.212:43255
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786289732823074-3431-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786289732823074-3431-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786289732823074-3431-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 15:35:39.804291 5031 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 15:35:39.804529 5031 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 15:35:39.804610 5031 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 15:35:39.809108 5031 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 15:35:39.809170 5031 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 15:35:39.809242 5031 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260809 15:35:39.809301 5031 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.3.89.195
I20260809 15:35:39.814486 5031 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.3.89.212:43255
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786289732823074-3431-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786289732823074-3431-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786289732823074-3431-0/krb5kdc/kudu_127.3.89.195.keytab
--principal=kudu/127.3.89.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.3.89.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786289732823074-3431-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.3.89.195
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.3.89.254:45563
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.5031
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.3.89.195
--log_dir=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786289732823074-3431-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 09 Aug 2026 14:46:55 UTC on ccycloud.quasar-yswkmf.root.comops.site
build id 4576540
I20260809 15:35:39.815555 5031 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 15:35:39.816581 5031 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 15:35:39.826292 5037 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 15:35:39.826552 5031 server_base.cc:1034] running on GCE node
W20260809 15:35:39.826293 5036 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 15:35:39.826292 5039 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 15:35:39.827142 5031 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 15:35:39.827767 5031 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 15:35:39.828950 5031 hybrid_clock.cc:648] HybridClock initialized: now 1786289739828925 us; error 47 us; skew 500 ppm
Aug 09 15:35:39 dist-test-slave-2w3w krb5kdc[4599](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786289739, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.89.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 15:35:39.831990 5031 init.cc:377] Logged in from keytab as kudu/127.3.89.195@KRBTEST.COM (short username kudu)
I20260809 15:35:39.832830 5031 webserver.cc:460] Webserver started at http://127.3.89.195:37619/ using document root <none> and password file <none>
I20260809 15:35:39.833408 5031 fs_manager.cc:359] Metadata directory not provided
I20260809 15:35:39.833472 5031 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 15:35:39.833680 5031 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 15:35:39.835978 5031 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786289732823074-3431-0/minicluster-data/ts-2/data/instance:
uuid: "58a4886f5b7a4aff99ca6f4a635db238"
format_stamp: "Formatted at 2026-08-09 15:35:39 on dist-test-slave-2w3w"
server_key: "646106a8c8acc7ff2a61caf855d93dec"
server_key_iv: "ea2108e311d78537b18a34477f9dc3a4"
server_key_version: "encryptionkey@0"
I20260809 15:35:39.836570 5031 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786289732823074-3431-0/minicluster-data/ts-2/wal/instance:
uuid: "58a4886f5b7a4aff99ca6f4a635db238"
format_stamp: "Formatted at 2026-08-09 15:35:39 on dist-test-slave-2w3w"
server_key: "646106a8c8acc7ff2a61caf855d93dec"
server_key_iv: "ea2108e311d78537b18a34477f9dc3a4"
server_key_version: "encryptionkey@0"
I20260809 15:35:39.840139 5031 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.004s sys 0.000s
I20260809 15:35:39.842571 5048 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 15:35:39.843742 5031 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260809 15:35:39.843873 5031 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786289732823074-3431-0/minicluster-data/ts-2/data,/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786289732823074-3431-0/minicluster-data/ts-2/wal
uuid: "58a4886f5b7a4aff99ca6f4a635db238"
format_stamp: "Formatted at 2026-08-09 15:35:39 on dist-test-slave-2w3w"
server_key: "646106a8c8acc7ff2a61caf855d93dec"
server_key_iv: "ea2108e311d78537b18a34477f9dc3a4"
server_key_version: "encryptionkey@0"
I20260809 15:35:39.844003 5031 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786289732823074-3431-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786289732823074-3431-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786289732823074-3431-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 15:35:39.854514 5031 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 15:35:39.857278 5031 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 15:35:39.857473 5031 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 15:35:39.858856 5031 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 15:35:39.858908 5031 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 15:35:39.858969 5031 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 15:35:39.859042 5031 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 09 15:35:39 dist-test-slave-2w3w krb5kdc[4599](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786289739, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.89.195@KRBTEST.COM for kudu/127.3.89.254@KRBTEST.COM
I20260809 15:35:39.870074 5031 rpc_server.cc:307] RPC server started. Bound to: 127.3.89.195:46417
I20260809 15:35:39.870175 5174 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.3.89.195:46417 every 8 connection(s)
I20260809 15:35:39.871111 5031 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786289732823074-3431-0/minicluster-data/ts-2/data/info.pb
I20260809 15:35:39.872937 4701 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:35:39.859780 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.89.195:58397 (local address 127.3.89.254:45563)
0809 15:35:39.859913 (+ 133us) server_negotiation.cc:206] Beginning negotiation
0809 15:35:39.859917 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 15:35:39.860966 (+ 1049us) server_negotiation.cc:407] Connection header received
0809 15:35:39.862125 (+ 1159us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 15:35:39.862128 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 15:35:39.862190 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 15:35:39.862275 (+ 85us) server_negotiation.cc:226] Negotiated authn=SASL
0809 15:35:39.864010 (+ 1735us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:35:39.864637 (+ 627us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:35:39.865792 (+ 1155us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:35:39.866103 (+ 311us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:35:39.866218 (+ 115us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:35:39.868893 (+ 2675us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 15:35:39.868898 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 15:35:39.868901 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 15:35:39.868931 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0809 15:35:39.870638 (+ 1707us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 15:35:39.871283 (+ 645us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 15:35:39.871286 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 15:35:39.871287 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 15:35:39.871367 (+ 80us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 15:35:39.871801 (+ 434us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 15:35:39.871807 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 15:35:39.871810 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 15:35:39.872041 (+ 231us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 15:35:39.872155 (+ 114us) server_negotiation.cc:1035] Waiting for connection context
0809 15:35:39.872628 (+ 473us) server_negotiation.cc:299] Negotiation successful
0809 15:35:39.872752 (+ 124us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":49}
I20260809 15:35:39.873627 5059 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:35:39.860313 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.89.254:45563 (local address 127.3.89.195:58397)
0809 15:35:39.860820 (+ 507us) negotiation.cc:107] Waiting for socket to connect
0809 15:35:39.860860 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0809 15:35:39.861867 (+ 1007us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 15:35:39.862528 (+ 661us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 15:35:39.862541 (+ 13us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 15:35:39.863063 (+ 522us) client_negotiation.cc:189] Negotiated authn=SASL
0809 15:35:39.863796 (+ 733us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:35:39.863821 (+ 25us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:35:39.864770 (+ 949us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:35:39.864774 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:35:39.865656 (+ 882us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:35:39.865665 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:35:39.866267 (+ 602us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:35:39.866272 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:35:39.866439 (+ 167us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:35:39.867058 (+ 619us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 15:35:39.867073 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0809 15:35:39.868682 (+ 1609us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 15:35:39.870752 (+ 2070us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 15:35:39.870757 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 15:35:39.870770 (+ 13us) client_negotiation.cc:760] calling sasl_client_step()
0809 15:35:39.871145 (+ 375us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 15:35:39.871466 (+ 321us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 15:35:39.871470 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 15:35:39.871473 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0809 15:35:39.871650 (+ 177us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 15:35:39.872153 (+ 503us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 15:35:39.872160 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 15:35:39.872387 (+ 227us) client_negotiation.cc:769] Sending connection context
0809 15:35:39.872565 (+ 178us) client_negotiation.cc:240] Negotiation successful
0809 15:35:39.872870 (+ 305us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":322,"thread_start_us":138,"threads_started":1}
I20260809 15:35:39.874812 5175 heartbeater.cc:347] Connected to a master server at 127.3.89.254:45563
I20260809 15:35:39.874965 5175 heartbeater.cc:464] Registering TS with master...
I20260809 15:35:39.875484 5175 heartbeater.cc:511] Master 127.3.89.254:45563 requested a full tablet report, sending...
I20260809 15:35:39.876580 4651 ts_manager.cc:194] Registered new tserver with Master: 58a4886f5b7a4aff99ca6f4a635db238 (127.3.89.195:46417)
I20260809 15:35:39.877226 4651 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.3.89.195@KRBTEST.COM'} at 127.3.89.195:58397
I20260809 15:35:39.881110 3431 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskm314YR/build/debug/bin/kudu as pid 5031
I20260809 15:35:39.881250 3431 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786289732823074-3431-0/minicluster-data/ts-2/wal/instance
I20260809 15:35:39.881537 3431 external_mini_cluster.cc:1428] Setting key 4e4b2c82e286edd5004be0d27ff317c6
I20260809 15:35:39.883627 3431 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260809 15:35:39.884382 4701 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:35:39.877486 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.89.195:45219 (local address 127.3.89.254:45563)
0809 15:35:39.877602 (+ 116us) server_negotiation.cc:206] Beginning negotiation
0809 15:35:39.877605 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0809 15:35:39.877616 (+ 11us) server_negotiation.cc:407] Connection header received
0809 15:35:39.877644 (+ 28us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 15:35:39.877646 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 15:35:39.877676 (+ 30us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 15:35:39.877769 (+ 93us) server_negotiation.cc:226] Negotiated authn=SASL
0809 15:35:39.878195 (+ 426us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:35:39.878682 (+ 487us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:35:39.879547 (+ 865us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:35:39.879758 (+ 211us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:35:39.879865 (+ 107us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:35:39.880949 (+ 1084us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 15:35:39.880952 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 15:35:39.880955 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 15:35:39.880978 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0809 15:35:39.882616 (+ 1638us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 15:35:39.883089 (+ 473us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 15:35:39.883092 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 15:35:39.883094 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 15:35:39.883176 (+ 82us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 15:35:39.883451 (+ 275us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 15:35:39.883455 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 15:35:39.883457 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 15:35:39.883671 (+ 214us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 15:35:39.883771 (+ 100us) server_negotiation.cc:1035] Waiting for connection context
0809 15:35:39.884132 (+ 361us) server_negotiation.cc:299] Negotiation successful
0809 15:35:39.884235 (+ 103us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":49}
I20260809 15:35:39.884420 5181 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:35:39.876876 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.89.254:45563 (local address 127.3.89.195:45219)
0809 15:35:39.877145 (+ 269us) negotiation.cc:107] Waiting for socket to connect
0809 15:35:39.877156 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0809 15:35:39.877222 (+ 66us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 15:35:39.877786 (+ 564us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 15:35:39.877789 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 15:35:39.877987 (+ 198us) client_negotiation.cc:189] Negotiated authn=SASL
0809 15:35:39.878081 (+ 94us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:35:39.878086 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:35:39.878804 (+ 718us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:35:39.878807 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:35:39.879429 (+ 622us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:35:39.879437 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:35:39.879873 (+ 436us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:35:39.879877 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:35:39.879965 (+ 88us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:35:39.880481 (+ 516us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 15:35:39.880496 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0809 15:35:39.880804 (+ 308us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 15:35:39.882720 (+ 1916us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 15:35:39.882724 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 15:35:39.882726 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 15:35:39.882951 (+ 225us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 15:35:39.883265 (+ 314us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 15:35:39.883269 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 15:35:39.883271 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 15:35:39.883358 (+ 87us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 15:35:39.883834 (+ 476us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 15:35:39.883837 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 15:35:39.883928 (+ 91us) client_negotiation.cc:769] Sending connection context
0809 15:35:39.884079 (+ 151us) client_negotiation.cc:240] Negotiation successful
0809 15:35:39.884222 (+ 143us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":175,"thread_start_us":107,"threads_started":1}
Aug 09 15:35:39 dist-test-slave-2w3w krb5kdc[4599](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786289739, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Aug 09 15:35:39 dist-test-slave-2w3w krb5kdc[4599](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786289739, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.3.89.254@KRBTEST.COM
I20260809 15:35:39.909766 4701 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:35:39.901572 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46748 (local address 127.3.89.254:45563)
0809 15:35:39.901723 (+ 151us) server_negotiation.cc:206] Beginning negotiation
0809 15:35:39.901726 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0809 15:35:39.901738 (+ 12us) server_negotiation.cc:407] Connection header received
0809 15:35:39.901773 (+ 35us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 15:35:39.901776 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 15:35:39.901813 (+ 37us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 15:35:39.901901 (+ 88us) server_negotiation.cc:226] Negotiated authn=SASL
0809 15:35:39.902589 (+ 688us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:35:39.903196 (+ 607us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:35:39.904030 (+ 834us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:35:39.904249 (+ 219us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:35:39.904354 (+ 105us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:35:39.906815 (+ 2461us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 15:35:39.906824 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 15:35:39.906826 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 15:35:39.906889 (+ 63us) server_negotiation.cc:922] calling sasl_server_start()
0809 15:35:39.908298 (+ 1409us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 15:35:39.908726 (+ 428us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 15:35:39.908729 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 15:35:39.908730 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 15:35:39.908799 (+ 69us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 15:35:39.909033 (+ 234us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 15:35:39.909035 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 15:35:39.909036 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 15:35:39.909187 (+ 151us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 15:35:39.909282 (+ 95us) server_negotiation.cc:1035] Waiting for connection context
0809 15:35:39.909479 (+ 197us) server_negotiation.cc:299] Negotiation successful
0809 15:35:39.909566 (+ 87us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":57}
I20260809 15:35:39.912530 4651 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:46748:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260809 15:35:39.915541 4651 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260809 15:35:39.930012 5197 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:35:39.923872 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.89.193:44379 (local address 127.0.0.1:45926)
0809 15:35:39.925341 (+ 1469us) negotiation.cc:107] Waiting for socket to connect
0809 15:35:39.925377 (+ 36us) client_negotiation.cc:174] Beginning negotiation
0809 15:35:39.925488 (+ 111us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 15:35:39.925994 (+ 506us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 15:35:39.926000 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 15:35:39.926022 (+ 22us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 15:35:39.926145 (+ 123us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:35:39.926151 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:35:39.927638 (+ 1487us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:35:39.927644 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:35:39.928690 (+ 1046us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:35:39.928699 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:35:39.929557 (+ 858us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:35:39.929560 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:35:39.929639 (+ 79us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:35:39.929668 (+ 29us) client_negotiation.cc:769] Sending connection context
0809 15:35:39.929739 (+ 71us) client_negotiation.cc:240] Negotiation successful
0809 15:35:39.929818 (+ 79us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1407,"mutex_wait_us":136,"thread_start_us":704,"threads_started":1}
I20260809 15:35:39.930038 5194 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:35:39.924054 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45926 (local address 127.3.89.193:44379)
0809 15:35:39.924406 (+ 352us) server_negotiation.cc:206] Beginning negotiation
0809 15:35:39.924423 (+ 17us) server_negotiation.cc:399] Waiting for connection header
0809 15:35:39.925468 (+ 1045us) server_negotiation.cc:407] Connection header received
0809 15:35:39.925609 (+ 141us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 15:35:39.925614 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 15:35:39.925825 (+ 211us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 15:35:39.925996 (+ 171us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 15:35:39.926279 (+ 283us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:35:39.927503 (+ 1224us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:35:39.928817 (+ 1314us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:35:39.929431 (+ 614us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:35:39.929554 (+ 123us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:35:39.929714 (+ 160us) server_negotiation.cc:1035] Waiting for connection context
0809 15:35:39.929782 (+ 68us) server_negotiation.cc:299] Negotiation successful
0809 15:35:39.929894 (+ 112us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":262,"thread_start_us":85,"threads_started":1}
I20260809 15:35:39.931548 5195 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:35:39.924597 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.89.194:44743 (local address 127.0.0.1:48422)
0809 15:35:39.925669 (+ 1072us) negotiation.cc:107] Waiting for socket to connect
0809 15:35:39.925682 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0809 15:35:39.925763 (+ 81us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 15:35:39.926286 (+ 523us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 15:35:39.926289 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 15:35:39.926304 (+ 15us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 15:35:39.926412 (+ 108us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:35:39.926424 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:35:39.927603 (+ 1179us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:35:39.927608 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:35:39.929200 (+ 1592us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:35:39.929212 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:35:39.931120 (+ 1908us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:35:39.931124 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:35:39.931210 (+ 86us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:35:39.931235 (+ 25us) client_negotiation.cc:769] Sending connection context
0809 15:35:39.931313 (+ 78us) client_negotiation.cc:240] Negotiation successful
0809 15:35:39.931420 (+ 107us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1016,"mutex_wait_us":131,"thread_start_us":95,"threads_started":1}
I20260809 15:35:39.931757 5196 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:35:39.923679 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.89.195:46417 (local address 127.0.0.1:48486)
0809 15:35:39.925247 (+ 1568us) negotiation.cc:107] Waiting for socket to connect
0809 15:35:39.925300 (+ 53us) client_negotiation.cc:174] Beginning negotiation
0809 15:35:39.925442 (+ 142us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 15:35:39.926137 (+ 695us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 15:35:39.926140 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 15:35:39.926157 (+ 17us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 15:35:39.926277 (+ 120us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:35:39.926284 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:35:39.927641 (+ 1357us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:35:39.927644 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:35:39.929200 (+ 1556us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:35:39.929212 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:35:39.931378 (+ 2166us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:35:39.931382 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:35:39.931477 (+ 95us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:35:39.931499 (+ 22us) client_negotiation.cc:769] Sending connection context
0809 15:35:39.931567 (+ 68us) client_negotiation.cc:240] Negotiation successful
0809 15:35:39.931634 (+ 67us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1385,"mutex_wait_us":187,"thread_start_us":863,"threads_started":1}
I20260809 15:35:39.931807 5193 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:35:39.923813 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48486 (local address 127.3.89.195:46417)
0809 15:35:39.924293 (+ 480us) server_negotiation.cc:206] Beginning negotiation
0809 15:35:39.924307 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0809 15:35:39.925412 (+ 1105us) server_negotiation.cc:407] Connection header received
0809 15:35:39.925627 (+ 215us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 15:35:39.925631 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 15:35:39.925821 (+ 190us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 15:35:39.925976 (+ 155us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 15:35:39.926406 (+ 430us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:35:39.927503 (+ 1097us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:35:39.930448 (+ 2945us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:35:39.931254 (+ 806us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:35:39.931377 (+ 123us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:35:39.931528 (+ 151us) server_negotiation.cc:1035] Waiting for connection context
0809 15:35:39.931594 (+ 66us) server_negotiation.cc:299] Negotiation successful
0809 15:35:39.931677 (+ 83us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":371,"thread_start_us":150,"threads_started":1}
I20260809 15:35:39.932093 5198 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:35:39.925025 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48422 (local address 127.3.89.194:44743)
0809 15:35:39.925354 (+ 329us) server_negotiation.cc:206] Beginning negotiation
0809 15:35:39.925366 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0809 15:35:39.925766 (+ 400us) server_negotiation.cc:407] Connection header received
0809 15:35:39.925851 (+ 85us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 15:35:39.925856 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 15:35:39.926016 (+ 160us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 15:35:39.926151 (+ 135us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 15:35:39.926523 (+ 372us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:35:39.927496 (+ 973us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:35:39.929335 (+ 1839us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:35:39.930147 (+ 812us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:35:39.930243 (+ 96us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:35:39.930390 (+ 147us) server_negotiation.cc:1035] Waiting for connection context
0809 15:35:39.931863 (+ 1473us) server_negotiation.cc:299] Negotiation successful
0809 15:35:39.931954 (+ 91us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":222,"thread_start_us":99,"threads_started":1}
I20260809 15:35:39.933398 4796 tablet_service.cc:1467] Processing CreateTablet for tablet 83dd86781b7343e387bdad3be322993a (DEFAULT_TABLE table=test-table [id=fe0aab481c5f497d9b5dcf191bbed5c7]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 15:35:39.934600 4796 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 83dd86781b7343e387bdad3be322993a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 15:35:39.934295 5105 tablet_service.cc:1467] Processing CreateTablet for tablet 83dd86781b7343e387bdad3be322993a (DEFAULT_TABLE table=test-table [id=fe0aab481c5f497d9b5dcf191bbed5c7]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 15:35:39.935294 5105 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 83dd86781b7343e387bdad3be322993a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 15:35:39.936363 4948 tablet_service.cc:1467] Processing CreateTablet for tablet 83dd86781b7343e387bdad3be322993a (DEFAULT_TABLE table=test-table [id=fe0aab481c5f497d9b5dcf191bbed5c7]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 15:35:39.937135 4948 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 83dd86781b7343e387bdad3be322993a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 15:35:39.942320 5199 tablet_bootstrap.cc:492] T 83dd86781b7343e387bdad3be322993a P 6383b28fd60f46e6b9627721662f1918: Bootstrap starting.
I20260809 15:35:39.944119 5200 tablet_bootstrap.cc:492] T 83dd86781b7343e387bdad3be322993a P 8ee7401f0286413092bba932ce0a7ba2: Bootstrap starting.
I20260809 15:35:39.944764 5201 tablet_bootstrap.cc:492] T 83dd86781b7343e387bdad3be322993a P 58a4886f5b7a4aff99ca6f4a635db238: Bootstrap starting.
I20260809 15:35:39.944787 5199 tablet_bootstrap.cc:654] T 83dd86781b7343e387bdad3be322993a P 6383b28fd60f46e6b9627721662f1918: Neither blocks nor log segments found. Creating new log.
I20260809 15:35:39.946286 5200 tablet_bootstrap.cc:654] T 83dd86781b7343e387bdad3be322993a P 8ee7401f0286413092bba932ce0a7ba2: Neither blocks nor log segments found. Creating new log.
I20260809 15:35:39.946449 5199 log.cc:824] T 83dd86781b7343e387bdad3be322993a P 6383b28fd60f46e6b9627721662f1918: Log is configured to *not* fsync() on all Append() calls
I20260809 15:35:39.947005 5201 tablet_bootstrap.cc:654] T 83dd86781b7343e387bdad3be322993a P 58a4886f5b7a4aff99ca6f4a635db238: Neither blocks nor log segments found. Creating new log.
I20260809 15:35:39.947180 5200 log.cc:824] T 83dd86781b7343e387bdad3be322993a P 8ee7401f0286413092bba932ce0a7ba2: Log is configured to *not* fsync() on all Append() calls
I20260809 15:35:39.947841 5201 log.cc:824] T 83dd86781b7343e387bdad3be322993a P 58a4886f5b7a4aff99ca6f4a635db238: Log is configured to *not* fsync() on all Append() calls
I20260809 15:35:39.948642 5199 tablet_bootstrap.cc:492] T 83dd86781b7343e387bdad3be322993a P 6383b28fd60f46e6b9627721662f1918: No bootstrap required, opened a new log
I20260809 15:35:39.948853 5199 ts_tablet_manager.cc:1397] T 83dd86781b7343e387bdad3be322993a P 6383b28fd60f46e6b9627721662f1918: Time spent bootstrapping tablet: real 0.007s user 0.005s sys 0.000s
I20260809 15:35:39.949196 5200 tablet_bootstrap.cc:492] T 83dd86781b7343e387bdad3be322993a P 8ee7401f0286413092bba932ce0a7ba2: No bootstrap required, opened a new log
I20260809 15:35:39.949378 5200 ts_tablet_manager.cc:1397] T 83dd86781b7343e387bdad3be322993a P 8ee7401f0286413092bba932ce0a7ba2: Time spent bootstrapping tablet: real 0.005s user 0.004s sys 0.000s
I20260809 15:35:39.949746 5201 tablet_bootstrap.cc:492] T 83dd86781b7343e387bdad3be322993a P 58a4886f5b7a4aff99ca6f4a635db238: No bootstrap required, opened a new log
I20260809 15:35:39.949923 5201 ts_tablet_manager.cc:1397] T 83dd86781b7343e387bdad3be322993a P 58a4886f5b7a4aff99ca6f4a635db238: Time spent bootstrapping tablet: real 0.005s user 0.003s sys 0.000s
I20260809 15:35:39.952283 5200 raft_consensus.cc:348] T 83dd86781b7343e387bdad3be322993a P 8ee7401f0286413092bba932ce0a7ba2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "58a4886f5b7a4aff99ca6f4a635db238" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 46417 } } peers { permanent_uuid: "6383b28fd60f46e6b9627721662f1918" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 44379 } } peers { permanent_uuid: "8ee7401f0286413092bba932ce0a7ba2" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 44743 } }
I20260809 15:35:39.952546 5200 raft_consensus.cc:374] T 83dd86781b7343e387bdad3be322993a P 8ee7401f0286413092bba932ce0a7ba2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 15:35:39.952610 5200 raft_consensus.cc:729] T 83dd86781b7343e387bdad3be322993a P 8ee7401f0286413092bba932ce0a7ba2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8ee7401f0286413092bba932ce0a7ba2, State: Initialized, Role: FOLLOWER
I20260809 15:35:39.952764 5199 raft_consensus.cc:348] T 83dd86781b7343e387bdad3be322993a P 6383b28fd60f46e6b9627721662f1918 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "58a4886f5b7a4aff99ca6f4a635db238" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 46417 } } peers { permanent_uuid: "6383b28fd60f46e6b9627721662f1918" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 44379 } } peers { permanent_uuid: "8ee7401f0286413092bba932ce0a7ba2" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 44743 } }
I20260809 15:35:39.953014 5199 raft_consensus.cc:374] T 83dd86781b7343e387bdad3be322993a P 6383b28fd60f46e6b9627721662f1918 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 15:35:39.953059 5199 raft_consensus.cc:729] T 83dd86781b7343e387bdad3be322993a P 6383b28fd60f46e6b9627721662f1918 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6383b28fd60f46e6b9627721662f1918, State: Initialized, Role: FOLLOWER
I20260809 15:35:39.953083 5200 consensus_queue.cc:260] T 83dd86781b7343e387bdad3be322993a P 8ee7401f0286413092bba932ce0a7ba2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "58a4886f5b7a4aff99ca6f4a635db238" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 46417 } } peers { permanent_uuid: "6383b28fd60f46e6b9627721662f1918" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 44379 } } peers { permanent_uuid: "8ee7401f0286413092bba932ce0a7ba2" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 44743 } }
I20260809 15:35:39.953451 5199 consensus_queue.cc:260] T 83dd86781b7343e387bdad3be322993a P 6383b28fd60f46e6b9627721662f1918 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "58a4886f5b7a4aff99ca6f4a635db238" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 46417 } } peers { permanent_uuid: "6383b28fd60f46e6b9627721662f1918" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 44379 } } peers { permanent_uuid: "8ee7401f0286413092bba932ce0a7ba2" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 44743 } }
I20260809 15:35:39.953701 5201 raft_consensus.cc:348] T 83dd86781b7343e387bdad3be322993a P 58a4886f5b7a4aff99ca6f4a635db238 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "58a4886f5b7a4aff99ca6f4a635db238" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 46417 } } peers { permanent_uuid: "6383b28fd60f46e6b9627721662f1918" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 44379 } } peers { permanent_uuid: "8ee7401f0286413092bba932ce0a7ba2" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 44743 } }
I20260809 15:35:39.953940 5201 raft_consensus.cc:374] T 83dd86781b7343e387bdad3be322993a P 58a4886f5b7a4aff99ca6f4a635db238 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 15:35:39.953994 5201 raft_consensus.cc:729] T 83dd86781b7343e387bdad3be322993a P 58a4886f5b7a4aff99ca6f4a635db238 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 58a4886f5b7a4aff99ca6f4a635db238, State: Initialized, Role: FOLLOWER
I20260809 15:35:39.954106 5022 heartbeater.cc:503] Master 127.3.89.254:45563 was elected leader, sending a full tablet report...
I20260809 15:35:39.954108 5200 ts_tablet_manager.cc:1428] T 83dd86781b7343e387bdad3be322993a P 8ee7401f0286413092bba932ce0a7ba2: Time spent starting tablet: real 0.005s user 0.006s sys 0.000s
I20260809 15:35:39.954402 5199 ts_tablet_manager.cc:1428] T 83dd86781b7343e387bdad3be322993a P 6383b28fd60f46e6b9627721662f1918: Time spent starting tablet: real 0.005s user 0.006s sys 0.000s
I20260809 15:35:39.954402 5201 consensus_queue.cc:260] T 83dd86781b7343e387bdad3be322993a P 58a4886f5b7a4aff99ca6f4a635db238 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "58a4886f5b7a4aff99ca6f4a635db238" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 46417 } } peers { permanent_uuid: "6383b28fd60f46e6b9627721662f1918" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 44379 } } peers { permanent_uuid: "8ee7401f0286413092bba932ce0a7ba2" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 44743 } }
I20260809 15:35:39.954708 4867 heartbeater.cc:503] Master 127.3.89.254:45563 was elected leader, sending a full tablet report...
I20260809 15:35:39.955725 5175 heartbeater.cc:503] Master 127.3.89.254:45563 was elected leader, sending a full tablet report...
I20260809 15:35:39.956015 5201 ts_tablet_manager.cc:1428] T 83dd86781b7343e387bdad3be322993a P 58a4886f5b7a4aff99ca6f4a635db238: Time spent starting tablet: real 0.006s user 0.006s sys 0.000s
I20260809 15:35:39.986611 5208 raft_consensus.cc:482] T 83dd86781b7343e387bdad3be322993a P 58a4886f5b7a4aff99ca6f4a635db238 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 15:35:39.986814 5208 raft_consensus.cc:504] T 83dd86781b7343e387bdad3be322993a P 58a4886f5b7a4aff99ca6f4a635db238 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "58a4886f5b7a4aff99ca6f4a635db238" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 46417 } } peers { permanent_uuid: "6383b28fd60f46e6b9627721662f1918" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 44379 } } peers { permanent_uuid: "8ee7401f0286413092bba932ce0a7ba2" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 44743 } }
I20260809 15:35:39.987876 5208 leader_election.cc:288] T 83dd86781b7343e387bdad3be322993a P 58a4886f5b7a4aff99ca6f4a635db238 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6383b28fd60f46e6b9627721662f1918 (127.3.89.193:44379), 8ee7401f0286413092bba932ce0a7ba2 (127.3.89.194:44743)
I20260809 15:35:39.992467 5194 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:35:39.988238 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.89.195:58767 (local address 127.3.89.193:44379)
0809 15:35:39.988358 (+ 120us) server_negotiation.cc:206] Beginning negotiation
0809 15:35:39.988363 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 15:35:39.988443 (+ 80us) server_negotiation.cc:407] Connection header received
0809 15:35:39.988585 (+ 142us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 15:35:39.988588 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 15:35:39.988656 (+ 68us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 15:35:39.988740 (+ 84us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 15:35:39.989037 (+ 297us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:35:39.989910 (+ 873us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:35:39.991329 (+ 1419us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:35:39.991936 (+ 607us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:35:39.992070 (+ 134us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:35:39.992121 (+ 51us) server_negotiation.cc:1035] Waiting for connection context
0809 15:35:39.992310 (+ 189us) server_negotiation.cc:299] Negotiation successful
0809 15:35:39.992368 (+ 58us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":47}
I20260809 15:35:39.992501 5059 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:35:39.988143 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.89.193:44379 (local address 127.3.89.195:58767)
0809 15:35:39.988355 (+ 212us) negotiation.cc:107] Waiting for socket to connect
0809 15:35:39.988371 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0809 15:35:39.988474 (+ 103us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 15:35:39.988764 (+ 290us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 15:35:39.988767 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 15:35:39.988786 (+ 19us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 15:35:39.988916 (+ 130us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:35:39.988926 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:35:39.990021 (+ 1095us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:35:39.990025 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:35:39.991192 (+ 1167us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:35:39.991202 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:35:39.992067 (+ 865us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:35:39.992071 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:35:39.992183 (+ 112us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:35:39.992208 (+ 25us) client_negotiation.cc:769] Sending connection context
0809 15:35:39.992290 (+ 82us) client_negotiation.cc:240] Negotiation successful
0809 15:35:39.992372 (+ 82us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":110}
I20260809 15:35:39.993114 4816 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "83dd86781b7343e387bdad3be322993a" candidate_uuid: "58a4886f5b7a4aff99ca6f4a635db238" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6383b28fd60f46e6b9627721662f1918" is_pre_election: true
I20260809 15:35:39.993218 5198 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:35:39.988237 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.89.195:49905 (local address 127.3.89.194:44743)
0809 15:35:39.988359 (+ 122us) server_negotiation.cc:206] Beginning negotiation
0809 15:35:39.988363 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 15:35:39.988736 (+ 373us) server_negotiation.cc:407] Connection header received
0809 15:35:39.988801 (+ 65us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 15:35:39.988804 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 15:35:39.988863 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 15:35:39.988950 (+ 87us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 15:35:39.989949 (+ 999us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:35:39.990715 (+ 766us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:35:39.992060 (+ 1345us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:35:39.992661 (+ 601us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:35:39.992788 (+ 127us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:35:39.992843 (+ 55us) server_negotiation.cc:1035] Waiting for connection context
0809 15:35:39.993046 (+ 203us) server_negotiation.cc:299] Negotiation successful
0809 15:35:39.993114 (+ 68us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":47}
I20260809 15:35:39.993229 5209 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:35:39.988143 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.89.194:44743 (local address 127.3.89.195:49905)
0809 15:35:39.988596 (+ 453us) negotiation.cc:107] Waiting for socket to connect
0809 15:35:39.988610 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0809 15:35:39.988729 (+ 119us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 15:35:39.988966 (+ 237us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 15:35:39.988968 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 15:35:39.989002 (+ 34us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 15:35:39.989820 (+ 818us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:35:39.989832 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:35:39.990836 (+ 1004us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:35:39.990839 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:35:39.991933 (+ 1094us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:35:39.991943 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:35:39.992796 (+ 853us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:35:39.992799 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:35:39.992909 (+ 110us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:35:39.992927 (+ 18us) client_negotiation.cc:769] Sending connection context
0809 15:35:39.993031 (+ 104us) client_negotiation.cc:240] Negotiation successful
0809 15:35:39.993097 (+ 66us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":385,"mutex_wait_us":35,"thread_start_us":120,"threads_started":1}
I20260809 15:35:39.993441 4816 raft_consensus.cc:2454] T 83dd86781b7343e387bdad3be322993a P 6383b28fd60f46e6b9627721662f1918 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 58a4886f5b7a4aff99ca6f4a635db238 in term 0.
I20260809 15:35:39.993968 5054 leader_election.cc:302] T 83dd86781b7343e387bdad3be322993a P 58a4886f5b7a4aff99ca6f4a635db238 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 58a4886f5b7a4aff99ca6f4a635db238, 6383b28fd60f46e6b9627721662f1918; no voters:
I20260809 15:35:39.993966 4970 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "83dd86781b7343e387bdad3be322993a" candidate_uuid: "58a4886f5b7a4aff99ca6f4a635db238" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8ee7401f0286413092bba932ce0a7ba2" is_pre_election: true
I20260809 15:35:39.994215 4970 raft_consensus.cc:2454] T 83dd86781b7343e387bdad3be322993a P 8ee7401f0286413092bba932ce0a7ba2 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 58a4886f5b7a4aff99ca6f4a635db238 in term 0.
I20260809 15:35:39.994241 5208 raft_consensus.cc:2781] T 83dd86781b7343e387bdad3be322993a P 58a4886f5b7a4aff99ca6f4a635db238 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260809 15:35:39.994324 5208 raft_consensus.cc:482] T 83dd86781b7343e387bdad3be322993a P 58a4886f5b7a4aff99ca6f4a635db238 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 15:35:39.994354 5208 raft_consensus.cc:3037] T 83dd86781b7343e387bdad3be322993a P 58a4886f5b7a4aff99ca6f4a635db238 [term 0 FOLLOWER]: Advancing to term 1
I20260809 15:35:39.995507 5208 raft_consensus.cc:504] T 83dd86781b7343e387bdad3be322993a P 58a4886f5b7a4aff99ca6f4a635db238 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "58a4886f5b7a4aff99ca6f4a635db238" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 46417 } } peers { permanent_uuid: "6383b28fd60f46e6b9627721662f1918" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 44379 } } peers { permanent_uuid: "8ee7401f0286413092bba932ce0a7ba2" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 44743 } }
I20260809 15:35:39.995841 5208 leader_election.cc:288] T 83dd86781b7343e387bdad3be322993a P 58a4886f5b7a4aff99ca6f4a635db238 [CANDIDATE]: Term 1 election: Requested vote from peers 6383b28fd60f46e6b9627721662f1918 (127.3.89.193:44379), 8ee7401f0286413092bba932ce0a7ba2 (127.3.89.194:44743)
I20260809 15:35:39.996191 4970 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "83dd86781b7343e387bdad3be322993a" candidate_uuid: "58a4886f5b7a4aff99ca6f4a635db238" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8ee7401f0286413092bba932ce0a7ba2"
I20260809 15:35:39.996196 4816 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "83dd86781b7343e387bdad3be322993a" candidate_uuid: "58a4886f5b7a4aff99ca6f4a635db238" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6383b28fd60f46e6b9627721662f1918"
I20260809 15:35:39.996308 4970 raft_consensus.cc:3037] T 83dd86781b7343e387bdad3be322993a P 8ee7401f0286413092bba932ce0a7ba2 [term 0 FOLLOWER]: Advancing to term 1
I20260809 15:35:39.996308 4816 raft_consensus.cc:3037] T 83dd86781b7343e387bdad3be322993a P 6383b28fd60f46e6b9627721662f1918 [term 0 FOLLOWER]: Advancing to term 1
I20260809 15:35:39.997709 4816 raft_consensus.cc:2454] T 83dd86781b7343e387bdad3be322993a P 6383b28fd60f46e6b9627721662f1918 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 58a4886f5b7a4aff99ca6f4a635db238 in term 1.
I20260809 15:35:39.997992 5054 leader_election.cc:302] T 83dd86781b7343e387bdad3be322993a P 58a4886f5b7a4aff99ca6f4a635db238 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 58a4886f5b7a4aff99ca6f4a635db238, 6383b28fd60f46e6b9627721662f1918; no voters:
I20260809 15:35:39.998186 5208 raft_consensus.cc:2781] T 83dd86781b7343e387bdad3be322993a P 58a4886f5b7a4aff99ca6f4a635db238 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 15:35:39.998214 4970 raft_consensus.cc:2454] T 83dd86781b7343e387bdad3be322993a P 8ee7401f0286413092bba932ce0a7ba2 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 58a4886f5b7a4aff99ca6f4a635db238 in term 1.
I20260809 15:35:39.998410 5208 raft_consensus.cc:686] T 83dd86781b7343e387bdad3be322993a P 58a4886f5b7a4aff99ca6f4a635db238 [term 1 LEADER]: Becoming Leader. State: Replica: 58a4886f5b7a4aff99ca6f4a635db238, State: Running, Role: LEADER
I20260809 15:35:39.998741 5208 consensus_queue.cc:237] T 83dd86781b7343e387bdad3be322993a P 58a4886f5b7a4aff99ca6f4a635db238 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "58a4886f5b7a4aff99ca6f4a635db238" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 46417 } } peers { permanent_uuid: "6383b28fd60f46e6b9627721662f1918" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 44379 } } peers { permanent_uuid: "8ee7401f0286413092bba932ce0a7ba2" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 44743 } }
I20260809 15:35:40.002128 4649 catalog_manager.cc:5495] T 83dd86781b7343e387bdad3be322993a P 58a4886f5b7a4aff99ca6f4a635db238 reported cstate change: term changed from 0 to 1, leader changed from <none> to 58a4886f5b7a4aff99ca6f4a635db238 (127.3.89.195). New cstate: current_term: 1 leader_uuid: "58a4886f5b7a4aff99ca6f4a635db238" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "58a4886f5b7a4aff99ca6f4a635db238" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 46417 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6383b28fd60f46e6b9627721662f1918" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 44379 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8ee7401f0286413092bba932ce0a7ba2" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 44743 } health_report { overall_health: UNKNOWN } } }
I20260809 15:35:40.036320 5193 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:35:40.032427 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48490 (local address 127.3.89.195:46417)
0809 15:35:40.032562 (+ 135us) server_negotiation.cc:206] Beginning negotiation
0809 15:35:40.032569 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 15:35:40.032585 (+ 16us) server_negotiation.cc:407] Connection header received
0809 15:35:40.032835 (+ 250us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 15:35:40.032840 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 15:35:40.032907 (+ 67us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 15:35:40.033019 (+ 112us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 15:35:40.033313 (+ 294us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:35:40.033814 (+ 501us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:35:40.034742 (+ 928us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:35:40.035005 (+ 263us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:35:40.035126 (+ 121us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:35:40.035459 (+ 333us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 15:35:40.035893 (+ 434us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 15:35:40.036023 (+ 130us) server_negotiation.cc:1035] Waiting for connection context
0809 15:35:40.036103 (+ 80us) server_negotiation.cc:299] Negotiation successful
0809 15:35:40.036160 (+ 57us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":45}
I20260809 15:35:40.042335 4970 raft_consensus.cc:1261] T 83dd86781b7343e387bdad3be322993a P 8ee7401f0286413092bba932ce0a7ba2 [term 1 FOLLOWER]: Refusing update from remote peer 58a4886f5b7a4aff99ca6f4a635db238: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 15:35:40.043102 5210 consensus_queue.cc:1035] T 83dd86781b7343e387bdad3be322993a P 58a4886f5b7a4aff99ca6f4a635db238 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8ee7401f0286413092bba932ce0a7ba2" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 44743 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 15:35:40.044106 4816 raft_consensus.cc:1261] T 83dd86781b7343e387bdad3be322993a P 6383b28fd60f46e6b9627721662f1918 [term 1 FOLLOWER]: Refusing update from remote peer 58a4886f5b7a4aff99ca6f4a635db238: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 15:35:40.044719 5210 consensus_queue.cc:1035] T 83dd86781b7343e387bdad3be322993a P 58a4886f5b7a4aff99ca6f4a635db238 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6383b28fd60f46e6b9627721662f1918" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 44379 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 15:35:40.051266 5214 mvcc.cc:205] Tried to move back new op lower bound from 7316642775205285888 to 7316642775037898752. Current Snapshot: MvccSnapshot[applied={T|T < 7316642775205285888}]
I20260809 15:35:40.052644 5216 mvcc.cc:205] Tried to move back new op lower bound from 7316642775205285888 to 7316642775037898752. Current Snapshot: MvccSnapshot[applied={T|T < 7316642775205285888}]
I20260809 15:35:40.063417 4651 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:46748:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 15:35:40.063681 4651 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:46748:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 15:35:40.066289 4651 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 2d52ec43c6d145ccbb52cd484ca05989: Sending DeleteTablet for 3 replicas of tablet 83dd86781b7343e387bdad3be322993a
I20260809 15:35:40.067153 5105 tablet_service.cc:1514] Processing DeleteTablet for tablet 83dd86781b7343e387bdad3be322993a with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 15:35:40 UTC) from {username='kudu', principal='kudu/127.3.89.254@KRBTEST.COM'} at 127.0.0.1:48486
I20260809 15:35:40.067152 4796 tablet_service.cc:1514] Processing DeleteTablet for tablet 83dd86781b7343e387bdad3be322993a with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 15:35:40 UTC) from {username='kudu', principal='kudu/127.3.89.254@KRBTEST.COM'} at 127.0.0.1:45926
I20260809 15:35:40.067225 4948 tablet_service.cc:1514] Processing DeleteTablet for tablet 83dd86781b7343e387bdad3be322993a with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 15:35:40 UTC) from {username='kudu', principal='kudu/127.3.89.254@KRBTEST.COM'} at 127.0.0.1:48422
I20260809 15:35:40.067927 5226 tablet_replica.cc:330] T 83dd86781b7343e387bdad3be322993a P 8ee7401f0286413092bba932ce0a7ba2: stopping tablet replica
I20260809 15:35:40.068246 5226 raft_consensus.cc:2229] T 83dd86781b7343e387bdad3be322993a P 8ee7401f0286413092bba932ce0a7ba2 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 15:35:40.068540 5226 raft_consensus.cc:2258] T 83dd86781b7343e387bdad3be322993a P 8ee7401f0286413092bba932ce0a7ba2 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 15:35:40.068573 4651 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:46748:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260809 15:35:40.068775 5224 tablet_replica.cc:330] T 83dd86781b7343e387bdad3be322993a P 6383b28fd60f46e6b9627721662f1918: stopping tablet replica
W20260809 15:35:40.068944 4651 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260809 15:35:40.069069 5224 raft_consensus.cc:2229] T 83dd86781b7343e387bdad3be322993a P 6383b28fd60f46e6b9627721662f1918 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 15:35:40.069319 5225 tablet_replica.cc:330] T 83dd86781b7343e387bdad3be322993a P 58a4886f5b7a4aff99ca6f4a635db238: stopping tablet replica
I20260809 15:35:40.069342 5224 raft_consensus.cc:2258] T 83dd86781b7343e387bdad3be322993a P 6383b28fd60f46e6b9627721662f1918 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 15:35:40.069645 5225 raft_consensus.cc:2229] T 83dd86781b7343e387bdad3be322993a P 58a4886f5b7a4aff99ca6f4a635db238 [term 1 LEADER]: Raft consensus shutting down.
I20260809 15:35:40.070386 5225 raft_consensus.cc:2258] T 83dd86781b7343e387bdad3be322993a P 58a4886f5b7a4aff99ca6f4a635db238 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 15:35:40.070863 5226 ts_tablet_manager.cc:1905] T 83dd86781b7343e387bdad3be322993a P 8ee7401f0286413092bba932ce0a7ba2: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 15:35:40.071976 5225 ts_tablet_manager.cc:1905] T 83dd86781b7343e387bdad3be322993a P 58a4886f5b7a4aff99ca6f4a635db238: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 15:35:40.073634 5226 ts_tablet_manager.cc:1918] T 83dd86781b7343e387bdad3be322993a P 8ee7401f0286413092bba932ce0a7ba2: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 15:35:40.073738 5226 log.cc:1196] T 83dd86781b7343e387bdad3be322993a P 8ee7401f0286413092bba932ce0a7ba2: Deleting WAL directory at /tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786289732823074-3431-0/minicluster-data/ts-1/wal/wals/83dd86781b7343e387bdad3be322993a
I20260809 15:35:40.074074 5226 ts_tablet_manager.cc:1939] T 83dd86781b7343e387bdad3be322993a P 8ee7401f0286413092bba932ce0a7ba2: Deleting consensus metadata
I20260809 15:35:40.075202 5105 tablet_service.cc:1467] Processing CreateTablet for tablet f0554ad5e76b4f1a83ff77194fd3be39 (DEFAULT_TABLE table=test-table [id=843e9a7bd5c640d4b13021f9fd530640]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 15:35:40.075202 4796 tablet_service.cc:1467] Processing CreateTablet for tablet f0554ad5e76b4f1a83ff77194fd3be39 (DEFAULT_TABLE table=test-table [id=843e9a7bd5c640d4b13021f9fd530640]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 15:35:40.075388 4633 catalog_manager.cc:4841] TS 8ee7401f0286413092bba932ce0a7ba2 (127.3.89.194:44743): tablet 83dd86781b7343e387bdad3be322993a (table test-table [id=fe0aab481c5f497d9b5dcf191bbed5c7]) successfully deleted
I20260809 15:35:40.075449 4796 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f0554ad5e76b4f1a83ff77194fd3be39. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 15:35:40.075449 5105 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f0554ad5e76b4f1a83ff77194fd3be39. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 15:35:40.075937 4948 tablet_service.cc:1467] Processing CreateTablet for tablet f0554ad5e76b4f1a83ff77194fd3be39 (DEFAULT_TABLE table=test-table [id=843e9a7bd5c640d4b13021f9fd530640]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 15:35:40.076184 4948 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f0554ad5e76b4f1a83ff77194fd3be39. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 15:35:40.077598 5201 tablet_bootstrap.cc:492] T f0554ad5e76b4f1a83ff77194fd3be39 P 58a4886f5b7a4aff99ca6f4a635db238: Bootstrap starting.
I20260809 15:35:40.077783 5199 tablet_bootstrap.cc:492] T f0554ad5e76b4f1a83ff77194fd3be39 P 6383b28fd60f46e6b9627721662f1918: Bootstrap starting.
I20260809 15:35:40.078395 5225 ts_tablet_manager.cc:1918] T 83dd86781b7343e387bdad3be322993a P 58a4886f5b7a4aff99ca6f4a635db238: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 15:35:40.078531 5200 tablet_bootstrap.cc:492] T f0554ad5e76b4f1a83ff77194fd3be39 P 8ee7401f0286413092bba932ce0a7ba2: Bootstrap starting.
I20260809 15:35:40.078549 5225 log.cc:1196] T 83dd86781b7343e387bdad3be322993a P 58a4886f5b7a4aff99ca6f4a635db238: Deleting WAL directory at /tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786289732823074-3431-0/minicluster-data/ts-2/wal/wals/83dd86781b7343e387bdad3be322993a
I20260809 15:35:40.078601 5201 tablet_bootstrap.cc:654] T f0554ad5e76b4f1a83ff77194fd3be39 P 58a4886f5b7a4aff99ca6f4a635db238: Neither blocks nor log segments found. Creating new log.
I20260809 15:35:40.078821 5199 tablet_bootstrap.cc:654] T f0554ad5e76b4f1a83ff77194fd3be39 P 6383b28fd60f46e6b9627721662f1918: Neither blocks nor log segments found. Creating new log.
I20260809 15:35:40.079483 5200 tablet_bootstrap.cc:654] T f0554ad5e76b4f1a83ff77194fd3be39 P 8ee7401f0286413092bba932ce0a7ba2: Neither blocks nor log segments found. Creating new log.
I20260809 15:35:40.079708 5225 ts_tablet_manager.cc:1939] T 83dd86781b7343e387bdad3be322993a P 58a4886f5b7a4aff99ca6f4a635db238: Deleting consensus metadata
I20260809 15:35:40.080900 4632 catalog_manager.cc:4841] TS 58a4886f5b7a4aff99ca6f4a635db238 (127.3.89.195:46417): tablet 83dd86781b7343e387bdad3be322993a (table test-table [id=fe0aab481c5f497d9b5dcf191bbed5c7]) successfully deleted
I20260809 15:35:40.081229 5199 tablet_bootstrap.cc:492] T f0554ad5e76b4f1a83ff77194fd3be39 P 6383b28fd60f46e6b9627721662f1918: No bootstrap required, opened a new log
I20260809 15:35:40.081305 5224 ts_tablet_manager.cc:1905] T 83dd86781b7343e387bdad3be322993a P 6383b28fd60f46e6b9627721662f1918: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 15:35:40.081344 5199 ts_tablet_manager.cc:1397] T f0554ad5e76b4f1a83ff77194fd3be39 P 6383b28fd60f46e6b9627721662f1918: Time spent bootstrapping tablet: real 0.004s user 0.002s sys 0.000s
I20260809 15:35:40.081827 5199 raft_consensus.cc:348] T f0554ad5e76b4f1a83ff77194fd3be39 P 6383b28fd60f46e6b9627721662f1918 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6383b28fd60f46e6b9627721662f1918" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 44379 } } peers { permanent_uuid: "8ee7401f0286413092bba932ce0a7ba2" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 44743 } } peers { permanent_uuid: "58a4886f5b7a4aff99ca6f4a635db238" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 46417 } }
I20260809 15:35:40.081948 5199 raft_consensus.cc:374] T f0554ad5e76b4f1a83ff77194fd3be39 P 6383b28fd60f46e6b9627721662f1918 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 15:35:40.081977 5200 tablet_bootstrap.cc:492] T f0554ad5e76b4f1a83ff77194fd3be39 P 8ee7401f0286413092bba932ce0a7ba2: No bootstrap required, opened a new log
I20260809 15:35:40.081983 5199 raft_consensus.cc:729] T f0554ad5e76b4f1a83ff77194fd3be39 P 6383b28fd60f46e6b9627721662f1918 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6383b28fd60f46e6b9627721662f1918, State: Initialized, Role: FOLLOWER
I20260809 15:35:40.082049 5200 ts_tablet_manager.cc:1397] T f0554ad5e76b4f1a83ff77194fd3be39 P 8ee7401f0286413092bba932ce0a7ba2: Time spent bootstrapping tablet: real 0.004s user 0.002s sys 0.000s
I20260809 15:35:40.082106 5199 consensus_queue.cc:260] T f0554ad5e76b4f1a83ff77194fd3be39 P 6383b28fd60f46e6b9627721662f1918 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6383b28fd60f46e6b9627721662f1918" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 44379 } } peers { permanent_uuid: "8ee7401f0286413092bba932ce0a7ba2" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 44743 } } peers { permanent_uuid: "58a4886f5b7a4aff99ca6f4a635db238" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 46417 } }
I20260809 15:35:40.082394 5199 ts_tablet_manager.cc:1428] T f0554ad5e76b4f1a83ff77194fd3be39 P 6383b28fd60f46e6b9627721662f1918: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260809 15:35:40.082480 5200 raft_consensus.cc:348] T f0554ad5e76b4f1a83ff77194fd3be39 P 8ee7401f0286413092bba932ce0a7ba2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6383b28fd60f46e6b9627721662f1918" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 44379 } } peers { permanent_uuid: "8ee7401f0286413092bba932ce0a7ba2" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 44743 } } peers { permanent_uuid: "58a4886f5b7a4aff99ca6f4a635db238" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 46417 } }
I20260809 15:35:40.082582 5200 raft_consensus.cc:374] T f0554ad5e76b4f1a83ff77194fd3be39 P 8ee7401f0286413092bba932ce0a7ba2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 15:35:40.082614 5200 raft_consensus.cc:729] T f0554ad5e76b4f1a83ff77194fd3be39 P 8ee7401f0286413092bba932ce0a7ba2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8ee7401f0286413092bba932ce0a7ba2, State: Initialized, Role: FOLLOWER
I20260809 15:35:40.082733 5200 consensus_queue.cc:260] T f0554ad5e76b4f1a83ff77194fd3be39 P 8ee7401f0286413092bba932ce0a7ba2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6383b28fd60f46e6b9627721662f1918" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 44379 } } peers { permanent_uuid: "8ee7401f0286413092bba932ce0a7ba2" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 44743 } } peers { permanent_uuid: "58a4886f5b7a4aff99ca6f4a635db238" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 46417 } }
I20260809 15:35:40.083019 5200 ts_tablet_manager.cc:1428] T f0554ad5e76b4f1a83ff77194fd3be39 P 8ee7401f0286413092bba932ce0a7ba2: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260809 15:35:40.084472 5224 ts_tablet_manager.cc:1918] T 83dd86781b7343e387bdad3be322993a P 6383b28fd60f46e6b9627721662f1918: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 15:35:40.084570 5224 log.cc:1196] T 83dd86781b7343e387bdad3be322993a P 6383b28fd60f46e6b9627721662f1918: Deleting WAL directory at /tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786289732823074-3431-0/minicluster-data/ts-0/wal/wals/83dd86781b7343e387bdad3be322993a
I20260809 15:35:40.084744 5201 tablet_bootstrap.cc:492] T f0554ad5e76b4f1a83ff77194fd3be39 P 58a4886f5b7a4aff99ca6f4a635db238: No bootstrap required, opened a new log
I20260809 15:35:40.084837 5201 ts_tablet_manager.cc:1397] T f0554ad5e76b4f1a83ff77194fd3be39 P 58a4886f5b7a4aff99ca6f4a635db238: Time spent bootstrapping tablet: real 0.007s user 0.002s sys 0.000s
I20260809 15:35:40.084923 5224 ts_tablet_manager.cc:1939] T 83dd86781b7343e387bdad3be322993a P 6383b28fd60f46e6b9627721662f1918: Deleting consensus metadata
I20260809 15:35:40.085390 5201 raft_consensus.cc:348] T f0554ad5e76b4f1a83ff77194fd3be39 P 58a4886f5b7a4aff99ca6f4a635db238 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6383b28fd60f46e6b9627721662f1918" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 44379 } } peers { permanent_uuid: "8ee7401f0286413092bba932ce0a7ba2" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 44743 } } peers { permanent_uuid: "58a4886f5b7a4aff99ca6f4a635db238" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 46417 } }
I20260809 15:35:40.085510 5201 raft_consensus.cc:374] T f0554ad5e76b4f1a83ff77194fd3be39 P 58a4886f5b7a4aff99ca6f4a635db238 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 15:35:40.085546 5201 raft_consensus.cc:729] T f0554ad5e76b4f1a83ff77194fd3be39 P 58a4886f5b7a4aff99ca6f4a635db238 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 58a4886f5b7a4aff99ca6f4a635db238, State: Initialized, Role: FOLLOWER
I20260809 15:35:40.085675 5201 consensus_queue.cc:260] T f0554ad5e76b4f1a83ff77194fd3be39 P 58a4886f5b7a4aff99ca6f4a635db238 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6383b28fd60f46e6b9627721662f1918" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 44379 } } peers { permanent_uuid: "8ee7401f0286413092bba932ce0a7ba2" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 44743 } } peers { permanent_uuid: "58a4886f5b7a4aff99ca6f4a635db238" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 46417 } }
I20260809 15:35:40.085943 5201 ts_tablet_manager.cc:1428] T f0554ad5e76b4f1a83ff77194fd3be39 P 58a4886f5b7a4aff99ca6f4a635db238: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260809 15:35:40.086423 4635 catalog_manager.cc:4841] TS 6383b28fd60f46e6b9627721662f1918 (127.3.89.193:44379): tablet 83dd86781b7343e387bdad3be322993a (table test-table [id=fe0aab481c5f497d9b5dcf191bbed5c7]) successfully deleted
I20260809 15:35:40.118472 5205 raft_consensus.cc:482] T f0554ad5e76b4f1a83ff77194fd3be39 P 8ee7401f0286413092bba932ce0a7ba2 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 15:35:40.118633 5205 raft_consensus.cc:504] T f0554ad5e76b4f1a83ff77194fd3be39 P 8ee7401f0286413092bba932ce0a7ba2 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6383b28fd60f46e6b9627721662f1918" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 44379 } } peers { permanent_uuid: "8ee7401f0286413092bba932ce0a7ba2" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 44743 } } peers { permanent_uuid: "58a4886f5b7a4aff99ca6f4a635db238" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 46417 } }
I20260809 15:35:40.119894 5205 leader_election.cc:288] T f0554ad5e76b4f1a83ff77194fd3be39 P 8ee7401f0286413092bba932ce0a7ba2 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6383b28fd60f46e6b9627721662f1918 (127.3.89.193:44379), 58a4886f5b7a4aff99ca6f4a635db238 (127.3.89.195:46417)
W20260809 15:35:40.122213 5176 tablet.cc:2369] T f0554ad5e76b4f1a83ff77194fd3be39 P 58a4886f5b7a4aff99ca6f4a635db238: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260809 15:35:40.123939 5193 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:35:40.120275 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.89.194:43255 (local address 127.3.89.195:46417)
0809 15:35:40.120400 (+ 125us) server_negotiation.cc:206] Beginning negotiation
0809 15:35:40.120405 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 15:35:40.120419 (+ 14us) server_negotiation.cc:407] Connection header received
0809 15:35:40.120477 (+ 58us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 15:35:40.120480 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 15:35:40.120526 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 15:35:40.120616 (+ 90us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 15:35:40.120932 (+ 316us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:35:40.121711 (+ 779us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:35:40.122989 (+ 1278us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:35:40.123393 (+ 404us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:35:40.123525 (+ 132us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:35:40.123582 (+ 57us) server_negotiation.cc:1035] Waiting for connection context
0809 15:35:40.123751 (+ 169us) server_negotiation.cc:299] Negotiation successful
0809 15:35:40.123822 (+ 71us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":45}
I20260809 15:35:40.123958 4902 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:35:40.120097 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.89.195:46417 (local address 127.3.89.194:43255)
0809 15:35:40.120294 (+ 197us) negotiation.cc:107] Waiting for socket to connect
0809 15:35:40.120310 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0809 15:35:40.120397 (+ 87us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 15:35:40.120627 (+ 230us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 15:35:40.120630 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 15:35:40.120655 (+ 25us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 15:35:40.120809 (+ 154us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:35:40.120816 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:35:40.121836 (+ 1020us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:35:40.121839 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:35:40.122828 (+ 989us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:35:40.122839 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:35:40.123515 (+ 676us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:35:40.123519 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:35:40.123619 (+ 100us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:35:40.123647 (+ 28us) client_negotiation.cc:769] Sending connection context
0809 15:35:40.123735 (+ 88us) client_negotiation.cc:240] Negotiation successful
0809 15:35:40.123813 (+ 78us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":111}
I20260809 15:35:40.124518 5125 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f0554ad5e76b4f1a83ff77194fd3be39" candidate_uuid: "8ee7401f0286413092bba932ce0a7ba2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "58a4886f5b7a4aff99ca6f4a635db238" is_pre_election: true
I20260809 15:35:40.124799 5125 raft_consensus.cc:2454] T f0554ad5e76b4f1a83ff77194fd3be39 P 58a4886f5b7a4aff99ca6f4a635db238 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8ee7401f0286413092bba932ce0a7ba2 in term 0.
I20260809 15:35:40.125233 4894 leader_election.cc:302] T f0554ad5e76b4f1a83ff77194fd3be39 P 8ee7401f0286413092bba932ce0a7ba2 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 58a4886f5b7a4aff99ca6f4a635db238, 8ee7401f0286413092bba932ce0a7ba2; no voters:
I20260809 15:35:40.125401 5231 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:35:40.120097 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.89.193:44379 (local address 127.3.89.194:40917)
0809 15:35:40.120663 (+ 566us) negotiation.cc:107] Waiting for socket to connect
0809 15:35:40.120675 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0809 15:35:40.120760 (+ 85us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 15:35:40.121151 (+ 391us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 15:35:40.121154 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 15:35:40.121169 (+ 15us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 15:35:40.121290 (+ 121us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:35:40.121295 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:35:40.122711 (+ 1416us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:35:40.122714 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:35:40.123989 (+ 1275us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:35:40.124000 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:35:40.124988 (+ 988us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:35:40.124992 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:35:40.125093 (+ 101us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:35:40.125114 (+ 21us) client_negotiation.cc:769] Sending connection context
0809 15:35:40.125185 (+ 71us) client_negotiation.cc:240] Negotiation successful
0809 15:35:40.125249 (+ 64us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":502,"mutex_wait_us":46,"thread_start_us":92,"threads_started":1}
I20260809 15:35:40.125411 5194 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:35:40.120786 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.89.194:40917 (local address 127.3.89.193:44379)
0809 15:35:40.120944 (+ 158us) server_negotiation.cc:206] Beginning negotiation
0809 15:35:40.120949 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 15:35:40.120961 (+ 12us) server_negotiation.cc:407] Connection header received
0809 15:35:40.121014 (+ 53us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 15:35:40.121016 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 15:35:40.121052 (+ 36us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 15:35:40.121139 (+ 87us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 15:35:40.121405 (+ 266us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:35:40.121971 (+ 566us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:35:40.124142 (+ 2171us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:35:40.124750 (+ 608us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:35:40.124839 (+ 89us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:35:40.124883 (+ 44us) server_negotiation.cc:1035] Waiting for connection context
0809 15:35:40.125202 (+ 319us) server_negotiation.cc:299] Negotiation successful
0809 15:35:40.125299 (+ 97us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":60}
I20260809 15:35:40.125499 5205 raft_consensus.cc:2781] T f0554ad5e76b4f1a83ff77194fd3be39 P 8ee7401f0286413092bba932ce0a7ba2 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260809 15:35:40.125548 5205 raft_consensus.cc:482] T f0554ad5e76b4f1a83ff77194fd3be39 P 8ee7401f0286413092bba932ce0a7ba2 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 15:35:40.125582 5205 raft_consensus.cc:3037] T f0554ad5e76b4f1a83ff77194fd3be39 P 8ee7401f0286413092bba932ce0a7ba2 [term 0 FOLLOWER]: Advancing to term 1
I20260809 15:35:40.125942 4816 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f0554ad5e76b4f1a83ff77194fd3be39" candidate_uuid: "8ee7401f0286413092bba932ce0a7ba2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6383b28fd60f46e6b9627721662f1918" is_pre_election: true
I20260809 15:35:40.126085 4816 raft_consensus.cc:2454] T f0554ad5e76b4f1a83ff77194fd3be39 P 6383b28fd60f46e6b9627721662f1918 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8ee7401f0286413092bba932ce0a7ba2 in term 0.
I20260809 15:35:40.126789 5205 raft_consensus.cc:504] T f0554ad5e76b4f1a83ff77194fd3be39 P 8ee7401f0286413092bba932ce0a7ba2 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6383b28fd60f46e6b9627721662f1918" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 44379 } } peers { permanent_uuid: "8ee7401f0286413092bba932ce0a7ba2" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 44743 } } peers { permanent_uuid: "58a4886f5b7a4aff99ca6f4a635db238" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 46417 } }
I20260809 15:35:40.127184 5205 leader_election.cc:288] T f0554ad5e76b4f1a83ff77194fd3be39 P 8ee7401f0286413092bba932ce0a7ba2 [CANDIDATE]: Term 1 election: Requested vote from peers 6383b28fd60f46e6b9627721662f1918 (127.3.89.193:44379), 58a4886f5b7a4aff99ca6f4a635db238 (127.3.89.195:46417)
I20260809 15:35:40.127526 5125 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f0554ad5e76b4f1a83ff77194fd3be39" candidate_uuid: "8ee7401f0286413092bba932ce0a7ba2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "58a4886f5b7a4aff99ca6f4a635db238"
I20260809 15:35:40.127646 5125 raft_consensus.cc:3037] T f0554ad5e76b4f1a83ff77194fd3be39 P 58a4886f5b7a4aff99ca6f4a635db238 [term 0 FOLLOWER]: Advancing to term 1
I20260809 15:35:40.127720 4816 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f0554ad5e76b4f1a83ff77194fd3be39" candidate_uuid: "8ee7401f0286413092bba932ce0a7ba2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6383b28fd60f46e6b9627721662f1918"
I20260809 15:35:40.127830 4816 raft_consensus.cc:3037] T f0554ad5e76b4f1a83ff77194fd3be39 P 6383b28fd60f46e6b9627721662f1918 [term 0 FOLLOWER]: Advancing to term 1
I20260809 15:35:40.128543 5125 raft_consensus.cc:2454] T f0554ad5e76b4f1a83ff77194fd3be39 P 58a4886f5b7a4aff99ca6f4a635db238 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8ee7401f0286413092bba932ce0a7ba2 in term 1.
I20260809 15:35:40.128796 4894 leader_election.cc:302] T f0554ad5e76b4f1a83ff77194fd3be39 P 8ee7401f0286413092bba932ce0a7ba2 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 58a4886f5b7a4aff99ca6f4a635db238, 8ee7401f0286413092bba932ce0a7ba2; no voters:
I20260809 15:35:40.128921 4816 raft_consensus.cc:2454] T f0554ad5e76b4f1a83ff77194fd3be39 P 6383b28fd60f46e6b9627721662f1918 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8ee7401f0286413092bba932ce0a7ba2 in term 1.
I20260809 15:35:40.128973 5205 raft_consensus.cc:2781] T f0554ad5e76b4f1a83ff77194fd3be39 P 8ee7401f0286413092bba932ce0a7ba2 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 15:35:40.129145 5205 raft_consensus.cc:686] T f0554ad5e76b4f1a83ff77194fd3be39 P 8ee7401f0286413092bba932ce0a7ba2 [term 1 LEADER]: Becoming Leader. State: Replica: 8ee7401f0286413092bba932ce0a7ba2, State: Running, Role: LEADER
I20260809 15:35:40.129427 5205 consensus_queue.cc:237] T f0554ad5e76b4f1a83ff77194fd3be39 P 8ee7401f0286413092bba932ce0a7ba2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6383b28fd60f46e6b9627721662f1918" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 44379 } } peers { permanent_uuid: "8ee7401f0286413092bba932ce0a7ba2" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 44743 } } peers { permanent_uuid: "58a4886f5b7a4aff99ca6f4a635db238" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 46417 } }
I20260809 15:35:40.131867 4649 catalog_manager.cc:5495] T f0554ad5e76b4f1a83ff77194fd3be39 P 8ee7401f0286413092bba932ce0a7ba2 reported cstate change: term changed from 0 to 1, leader changed from <none> to 8ee7401f0286413092bba932ce0a7ba2 (127.3.89.194). New cstate: current_term: 1 leader_uuid: "8ee7401f0286413092bba932ce0a7ba2" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6383b28fd60f46e6b9627721662f1918" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 44379 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8ee7401f0286413092bba932ce0a7ba2" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 44743 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "58a4886f5b7a4aff99ca6f4a635db238" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 46417 } health_report { overall_health: UNKNOWN } } }
W20260809 15:35:40.159029 5023 tablet.cc:2369] T f0554ad5e76b4f1a83ff77194fd3be39 P 8ee7401f0286413092bba932ce0a7ba2: Can't schedule compaction. Clean time has not been advanced past its initial value.
Aug 09 15:35:40 dist-test-slave-2w3w krb5kdc[4599](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786289739, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.89.254@KRBTEST.COM for kudu/127.3.89.254@KRBTEST.COM
I20260809 15:35:40.181845 5240 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:35:40.172657 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.89.254:45563 (local address 127.0.0.1:46760)
0809 15:35:40.172951 (+ 294us) negotiation.cc:107] Waiting for socket to connect
0809 15:35:40.172968 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0809 15:35:40.173112 (+ 144us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 15:35:40.173356 (+ 244us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 15:35:40.173358 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 15:35:40.173735 (+ 377us) client_negotiation.cc:189] Negotiated authn=SASL
0809 15:35:40.173832 (+ 97us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:35:40.173838 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:35:40.174712 (+ 874us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:35:40.174730 (+ 18us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:35:40.175363 (+ 633us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:35:40.175373 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:35:40.175915 (+ 542us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:35:40.175918 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:35:40.176000 (+ 82us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:35:40.176616 (+ 616us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 15:35:40.176629 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0809 15:35:40.178049 (+ 1420us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 15:35:40.180174 (+ 2125us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 15:35:40.180186 (+ 12us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 15:35:40.180193 (+ 7us) client_negotiation.cc:760] calling sasl_client_step()
0809 15:35:40.180465 (+ 272us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 15:35:40.180748 (+ 283us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 15:35:40.180751 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 15:35:40.180753 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 15:35:40.180858 (+ 105us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 15:35:40.181270 (+ 412us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 15:35:40.181274 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 15:35:40.181361 (+ 87us) client_negotiation.cc:769] Sending connection context
0809 15:35:40.181471 (+ 110us) client_negotiation.cc:240] Negotiation successful
0809 15:35:40.181638 (+ 167us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":214,"thread_start_us":134,"threads_started":1}
I20260809 15:35:40.181845 4701 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:35:40.172666 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46760 (local address 127.3.89.254:45563)
0809 15:35:40.172790 (+ 124us) server_negotiation.cc:206] Beginning negotiation
0809 15:35:40.172795 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 15:35:40.173053 (+ 258us) server_negotiation.cc:407] Connection header received
0809 15:35:40.173212 (+ 159us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 15:35:40.173216 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 15:35:40.173256 (+ 40us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 15:35:40.173339 (+ 83us) server_negotiation.cc:226] Negotiated authn=SASL
0809 15:35:40.173948 (+ 609us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:35:40.174590 (+ 642us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:35:40.175475 (+ 885us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:35:40.175775 (+ 300us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:35:40.175913 (+ 138us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:35:40.178182 (+ 2269us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 15:35:40.178186 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 15:35:40.178189 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 15:35:40.178214 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0809 15:35:40.180031 (+ 1817us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 15:35:40.180563 (+ 532us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 15:35:40.180568 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 15:35:40.180570 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 15:35:40.180651 (+ 81us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 15:35:40.180959 (+ 308us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 15:35:40.180963 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 15:35:40.180964 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 15:35:40.181160 (+ 196us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 15:35:40.181217 (+ 57us) server_negotiation.cc:1035] Waiting for connection context
0809 15:35:40.181517 (+ 300us) server_negotiation.cc:299] Negotiation successful
0809 15:35:40.181699 (+ 182us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":81,"server-negotiator.queue_time_us":42}
I20260809 15:35:40.186437 4649 catalog_manager.cc:1908] Servicing CreateTable request from {username='kudu', principal='kudu/127.3.89.254@KRBTEST.COM'} at 127.0.0.1:46760:
name: "kudu_system.kudu_transactions"
schema {
columns {
name: "txn_id"
type: INT64
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "entry_type"
type: INT8
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "identifier"
type: STRING
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "metadata"
type: STRING
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
rows: "<redacted>""\006\001\000\000\000\000\000\000\000\000\007\001@B\017\000\000\000\000\000"
indirect_data: "<redacted>"""
}
partition_schema {
range_schema {
columns {
name: "txn_id"
}
}
}
table_type: TXN_STATUS_TABLE
W20260809 15:35:40.187216 4649 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table kudu_system.kudu_transactions in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
W20260809 15:35:40.191272 4868 tablet.cc:2369] T f0554ad5e76b4f1a83ff77194fd3be39 P 6383b28fd60f46e6b9627721662f1918: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260809 15:35:40.192365 4796 tablet_service.cc:1467] Processing CreateTablet for tablet 3e2f2d410fc44708a2bb08bf05ef8ece (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=efdbd9afca614ca6ba3670ee6d4223ed]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260809 15:35:40.192447 5105 tablet_service.cc:1467] Processing CreateTablet for tablet 3e2f2d410fc44708a2bb08bf05ef8ece (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=efdbd9afca614ca6ba3670ee6d4223ed]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260809 15:35:40.192444 4948 tablet_service.cc:1467] Processing CreateTablet for tablet 3e2f2d410fc44708a2bb08bf05ef8ece (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=efdbd9afca614ca6ba3670ee6d4223ed]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260809 15:35:40.192749 4796 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 3e2f2d410fc44708a2bb08bf05ef8ece. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 15:35:40.192777 5105 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 3e2f2d410fc44708a2bb08bf05ef8ece. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 15:35:40.192840 4948 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 3e2f2d410fc44708a2bb08bf05ef8ece. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 15:35:40.195758 5199 tablet_bootstrap.cc:492] T 3e2f2d410fc44708a2bb08bf05ef8ece P 6383b28fd60f46e6b9627721662f1918: Bootstrap starting.
I20260809 15:35:40.196514 5199 tablet_bootstrap.cc:654] T 3e2f2d410fc44708a2bb08bf05ef8ece P 6383b28fd60f46e6b9627721662f1918: Neither blocks nor log segments found. Creating new log.
I20260809 15:35:40.196698 5200 tablet_bootstrap.cc:492] T 3e2f2d410fc44708a2bb08bf05ef8ece P 8ee7401f0286413092bba932ce0a7ba2: Bootstrap starting.
I20260809 15:35:40.197259 5201 tablet_bootstrap.cc:492] T 3e2f2d410fc44708a2bb08bf05ef8ece P 58a4886f5b7a4aff99ca6f4a635db238: Bootstrap starting.
I20260809 15:35:40.197650 5200 tablet_bootstrap.cc:654] T 3e2f2d410fc44708a2bb08bf05ef8ece P 8ee7401f0286413092bba932ce0a7ba2: Neither blocks nor log segments found. Creating new log.
I20260809 15:35:40.198053 5199 tablet_bootstrap.cc:492] T 3e2f2d410fc44708a2bb08bf05ef8ece P 6383b28fd60f46e6b9627721662f1918: No bootstrap required, opened a new log
I20260809 15:35:40.198143 5199 ts_tablet_manager.cc:1397] T 3e2f2d410fc44708a2bb08bf05ef8ece P 6383b28fd60f46e6b9627721662f1918: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260809 15:35:40.198650 5201 tablet_bootstrap.cc:654] T 3e2f2d410fc44708a2bb08bf05ef8ece P 58a4886f5b7a4aff99ca6f4a635db238: Neither blocks nor log segments found. Creating new log.
I20260809 15:35:40.198576 5199 raft_consensus.cc:348] T 3e2f2d410fc44708a2bb08bf05ef8ece P 6383b28fd60f46e6b9627721662f1918 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6383b28fd60f46e6b9627721662f1918" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 44379 } } peers { permanent_uuid: "8ee7401f0286413092bba932ce0a7ba2" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 44743 } } peers { permanent_uuid: "58a4886f5b7a4aff99ca6f4a635db238" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 46417 } }
I20260809 15:35:40.198695 5199 raft_consensus.cc:374] T 3e2f2d410fc44708a2bb08bf05ef8ece P 6383b28fd60f46e6b9627721662f1918 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 15:35:40.198734 5199 raft_consensus.cc:729] T 3e2f2d410fc44708a2bb08bf05ef8ece P 6383b28fd60f46e6b9627721662f1918 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6383b28fd60f46e6b9627721662f1918, State: Initialized, Role: FOLLOWER
I20260809 15:35:40.198863 5199 consensus_queue.cc:260] T 3e2f2d410fc44708a2bb08bf05ef8ece P 6383b28fd60f46e6b9627721662f1918 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6383b28fd60f46e6b9627721662f1918" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 44379 } } peers { permanent_uuid: "8ee7401f0286413092bba932ce0a7ba2" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 44743 } } peers { permanent_uuid: "58a4886f5b7a4aff99ca6f4a635db238" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 46417 } }
I20260809 15:35:40.199189 5199 ts_tablet_manager.cc:1428] T 3e2f2d410fc44708a2bb08bf05ef8ece P 6383b28fd60f46e6b9627721662f1918: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260809 15:35:40.199489 5207 tablet_replica.cc:439] T 3e2f2d410fc44708a2bb08bf05ef8ece P 6383b28fd60f46e6b9627721662f1918: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6383b28fd60f46e6b9627721662f1918" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 44379 } } peers { permanent_uuid: "8ee7401f0286413092bba932ce0a7ba2" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 44743 } } peers { permanent_uuid: "58a4886f5b7a4aff99ca6f4a635db238" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 46417 } } }
I20260809 15:35:40.199631 5207 tablet_replica.cc:442] T 3e2f2d410fc44708a2bb08bf05ef8ece P 6383b28fd60f46e6b9627721662f1918: This TxnStatusTablet replica's current role is: FOLLOWER
I20260809 15:35:40.200395 5200 tablet_bootstrap.cc:492] T 3e2f2d410fc44708a2bb08bf05ef8ece P 8ee7401f0286413092bba932ce0a7ba2: No bootstrap required, opened a new log
I20260809 15:35:40.200481 5200 ts_tablet_manager.cc:1397] T 3e2f2d410fc44708a2bb08bf05ef8ece P 8ee7401f0286413092bba932ce0a7ba2: Time spent bootstrapping tablet: real 0.004s user 0.002s sys 0.000s
I20260809 15:35:40.200685 5201 tablet_bootstrap.cc:492] T 3e2f2d410fc44708a2bb08bf05ef8ece P 58a4886f5b7a4aff99ca6f4a635db238: No bootstrap required, opened a new log
I20260809 15:35:40.200774 5201 ts_tablet_manager.cc:1397] T 3e2f2d410fc44708a2bb08bf05ef8ece P 58a4886f5b7a4aff99ca6f4a635db238: Time spent bootstrapping tablet: real 0.004s user 0.002s sys 0.000s
I20260809 15:35:40.200903 5200 raft_consensus.cc:348] T 3e2f2d410fc44708a2bb08bf05ef8ece P 8ee7401f0286413092bba932ce0a7ba2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6383b28fd60f46e6b9627721662f1918" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 44379 } } peers { permanent_uuid: "8ee7401f0286413092bba932ce0a7ba2" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 44743 } } peers { permanent_uuid: "58a4886f5b7a4aff99ca6f4a635db238" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 46417 } }
I20260809 15:35:40.201022 5200 raft_consensus.cc:374] T 3e2f2d410fc44708a2bb08bf05ef8ece P 8ee7401f0286413092bba932ce0a7ba2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 15:35:40.201058 5200 raft_consensus.cc:729] T 3e2f2d410fc44708a2bb08bf05ef8ece P 8ee7401f0286413092bba932ce0a7ba2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8ee7401f0286413092bba932ce0a7ba2, State: Initialized, Role: FOLLOWER
I20260809 15:35:40.201177 5200 consensus_queue.cc:260] T 3e2f2d410fc44708a2bb08bf05ef8ece P 8ee7401f0286413092bba932ce0a7ba2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6383b28fd60f46e6b9627721662f1918" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 44379 } } peers { permanent_uuid: "8ee7401f0286413092bba932ce0a7ba2" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 44743 } } peers { permanent_uuid: "58a4886f5b7a4aff99ca6f4a635db238" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 46417 } }
I20260809 15:35:40.201222 5201 raft_consensus.cc:348] T 3e2f2d410fc44708a2bb08bf05ef8ece P 58a4886f5b7a4aff99ca6f4a635db238 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6383b28fd60f46e6b9627721662f1918" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 44379 } } peers { permanent_uuid: "8ee7401f0286413092bba932ce0a7ba2" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 44743 } } peers { permanent_uuid: "58a4886f5b7a4aff99ca6f4a635db238" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 46417 } }
I20260809 15:35:40.201334 5201 raft_consensus.cc:374] T 3e2f2d410fc44708a2bb08bf05ef8ece P 58a4886f5b7a4aff99ca6f4a635db238 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 15:35:40.201370 5201 raft_consensus.cc:729] T 3e2f2d410fc44708a2bb08bf05ef8ece P 58a4886f5b7a4aff99ca6f4a635db238 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 58a4886f5b7a4aff99ca6f4a635db238, State: Initialized, Role: FOLLOWER
I20260809 15:35:40.201440 5232 tablet_replica.cc:439] T 3e2f2d410fc44708a2bb08bf05ef8ece P 8ee7401f0286413092bba932ce0a7ba2: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6383b28fd60f46e6b9627721662f1918" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 44379 } } peers { permanent_uuid: "8ee7401f0286413092bba932ce0a7ba2" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 44743 } } peers { permanent_uuid: "58a4886f5b7a4aff99ca6f4a635db238" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 46417 } } }
I20260809 15:35:40.201491 5201 consensus_queue.cc:260] T 3e2f2d410fc44708a2bb08bf05ef8ece P 58a4886f5b7a4aff99ca6f4a635db238 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6383b28fd60f46e6b9627721662f1918" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 44379 } } peers { permanent_uuid: "8ee7401f0286413092bba932ce0a7ba2" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 44743 } } peers { permanent_uuid: "58a4886f5b7a4aff99ca6f4a635db238" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 46417 } }
I20260809 15:35:40.201575 5232 tablet_replica.cc:442] T 3e2f2d410fc44708a2bb08bf05ef8ece P 8ee7401f0286413092bba932ce0a7ba2: This TxnStatusTablet replica's current role is: FOLLOWER
I20260809 15:35:40.201733 5201 ts_tablet_manager.cc:1428] T 3e2f2d410fc44708a2bb08bf05ef8ece P 58a4886f5b7a4aff99ca6f4a635db238: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260809 15:35:40.201786 5220 tablet_replica.cc:439] T 3e2f2d410fc44708a2bb08bf05ef8ece P 58a4886f5b7a4aff99ca6f4a635db238: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6383b28fd60f46e6b9627721662f1918" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 44379 } } peers { permanent_uuid: "8ee7401f0286413092bba932ce0a7ba2" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 44743 } } peers { permanent_uuid: "58a4886f5b7a4aff99ca6f4a635db238" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 46417 } } }
I20260809 15:35:40.201910 5220 tablet_replica.cc:442] T 3e2f2d410fc44708a2bb08bf05ef8ece P 58a4886f5b7a4aff99ca6f4a635db238: This TxnStatusTablet replica's current role is: FOLLOWER
I20260809 15:35:40.201455 5200 ts_tablet_manager.cc:1428] T 3e2f2d410fc44708a2bb08bf05ef8ece P 8ee7401f0286413092bba932ce0a7ba2: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260809 15:35:40.268981 5207 raft_consensus.cc:482] T 3e2f2d410fc44708a2bb08bf05ef8ece P 6383b28fd60f46e6b9627721662f1918 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 15:35:40.269147 5207 raft_consensus.cc:504] T 3e2f2d410fc44708a2bb08bf05ef8ece P 6383b28fd60f46e6b9627721662f1918 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6383b28fd60f46e6b9627721662f1918" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 44379 } } peers { permanent_uuid: "8ee7401f0286413092bba932ce0a7ba2" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 44743 } } peers { permanent_uuid: "58a4886f5b7a4aff99ca6f4a635db238" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 46417 } }
I20260809 15:35:40.270205 5207 leader_election.cc:288] T 3e2f2d410fc44708a2bb08bf05ef8ece P 6383b28fd60f46e6b9627721662f1918 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 8ee7401f0286413092bba932ce0a7ba2 (127.3.89.194:44743), 58a4886f5b7a4aff99ca6f4a635db238 (127.3.89.195:46417)
I20260809 15:35:40.274443 5193 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:35:40.270567 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.89.193:33401 (local address 127.3.89.195:46417)
0809 15:35:40.270694 (+ 127us) server_negotiation.cc:206] Beginning negotiation
0809 15:35:40.270698 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 15:35:40.271145 (+ 447us) server_negotiation.cc:407] Connection header received
0809 15:35:40.271259 (+ 114us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 15:35:40.271261 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 15:35:40.271336 (+ 75us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 15:35:40.271434 (+ 98us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 15:35:40.271695 (+ 261us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:35:40.272173 (+ 478us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:35:40.273343 (+ 1170us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:35:40.273851 (+ 508us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:35:40.273979 (+ 128us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:35:40.274026 (+ 47us) server_negotiation.cc:1035] Waiting for connection context
0809 15:35:40.274223 (+ 197us) server_negotiation.cc:299] Negotiation successful
0809 15:35:40.274302 (+ 79us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":47}
I20260809 15:35:40.274462 5245 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:35:40.270473 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.89.195:46417 (local address 127.3.89.193:33401)
0809 15:35:40.271008 (+ 535us) negotiation.cc:107] Waiting for socket to connect
0809 15:35:40.271023 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0809 15:35:40.271160 (+ 137us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 15:35:40.271451 (+ 291us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 15:35:40.271453 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 15:35:40.271466 (+ 13us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 15:35:40.271585 (+ 119us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:35:40.271591 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:35:40.272315 (+ 724us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:35:40.272318 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:35:40.273192 (+ 874us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:35:40.273201 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:35:40.273980 (+ 779us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:35:40.273984 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:35:40.274100 (+ 116us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:35:40.274123 (+ 23us) client_negotiation.cc:769] Sending connection context
0809 15:35:40.274201 (+ 78us) client_negotiation.cc:240] Negotiation successful
0809 15:35:40.274294 (+ 93us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":425,"mutex_wait_us":100,"thread_start_us":110,"threads_started":1}
I20260809 15:35:40.274550 5198 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:35:40.270566 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.89.193:39761 (local address 127.3.89.194:44743)
0809 15:35:40.270694 (+ 128us) server_negotiation.cc:206] Beginning negotiation
0809 15:35:40.270698 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 15:35:40.271001 (+ 303us) server_negotiation.cc:407] Connection header received
0809 15:35:40.271151 (+ 150us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 15:35:40.271154 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 15:35:40.271223 (+ 69us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 15:35:40.271304 (+ 81us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 15:35:40.271623 (+ 319us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:35:40.272148 (+ 525us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:35:40.273302 (+ 1154us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:35:40.273841 (+ 539us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:35:40.273964 (+ 123us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:35:40.274018 (+ 54us) server_negotiation.cc:1035] Waiting for connection context
0809 15:35:40.274350 (+ 332us) server_negotiation.cc:299] Negotiation successful
0809 15:35:40.274419 (+ 69us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":56}
I20260809 15:35:40.274945 5244 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:35:40.270474 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.89.194:44743 (local address 127.3.89.193:39761)
0809 15:35:40.270859 (+ 385us) negotiation.cc:107] Waiting for socket to connect
0809 15:35:40.270871 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0809 15:35:40.271033 (+ 162us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 15:35:40.271328 (+ 295us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 15:35:40.271331 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 15:35:40.271346 (+ 15us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 15:35:40.271496 (+ 150us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:35:40.271506 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:35:40.272266 (+ 760us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:35:40.272271 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:35:40.273180 (+ 909us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:35:40.273189 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:35:40.274082 (+ 893us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:35:40.274085 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:35:40.274183 (+ 98us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:35:40.274208 (+ 25us) client_negotiation.cc:769] Sending connection context
0809 15:35:40.274680 (+ 472us) client_negotiation.cc:240] Negotiation successful
0809 15:35:40.274793 (+ 113us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":324,"mutex_wait_us":22,"thread_start_us":139,"threads_started":1}
I20260809 15:35:40.275059 5125 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3e2f2d410fc44708a2bb08bf05ef8ece" candidate_uuid: "6383b28fd60f46e6b9627721662f1918" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "58a4886f5b7a4aff99ca6f4a635db238" is_pre_election: true
I20260809 15:35:40.275221 5125 raft_consensus.cc:2454] T 3e2f2d410fc44708a2bb08bf05ef8ece P 58a4886f5b7a4aff99ca6f4a635db238 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6383b28fd60f46e6b9627721662f1918 in term 0.
I20260809 15:35:40.275457 4970 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3e2f2d410fc44708a2bb08bf05ef8ece" candidate_uuid: "6383b28fd60f46e6b9627721662f1918" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8ee7401f0286413092bba932ce0a7ba2" is_pre_election: true
I20260809 15:35:40.275597 4970 raft_consensus.cc:2454] T 3e2f2d410fc44708a2bb08bf05ef8ece P 8ee7401f0286413092bba932ce0a7ba2 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6383b28fd60f46e6b9627721662f1918 in term 0.
I20260809 15:35:40.275629 4741 leader_election.cc:302] T 3e2f2d410fc44708a2bb08bf05ef8ece P 6383b28fd60f46e6b9627721662f1918 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 58a4886f5b7a4aff99ca6f4a635db238, 6383b28fd60f46e6b9627721662f1918; no voters:
I20260809 15:35:40.275882 5207 raft_consensus.cc:2781] T 3e2f2d410fc44708a2bb08bf05ef8ece P 6383b28fd60f46e6b9627721662f1918 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260809 15:35:40.275949 5207 raft_consensus.cc:482] T 3e2f2d410fc44708a2bb08bf05ef8ece P 6383b28fd60f46e6b9627721662f1918 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 15:35:40.276008 5207 raft_consensus.cc:3037] T 3e2f2d410fc44708a2bb08bf05ef8ece P 6383b28fd60f46e6b9627721662f1918 [term 0 FOLLOWER]: Advancing to term 1
I20260809 15:35:40.277002 5207 raft_consensus.cc:504] T 3e2f2d410fc44708a2bb08bf05ef8ece P 6383b28fd60f46e6b9627721662f1918 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6383b28fd60f46e6b9627721662f1918" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 44379 } } peers { permanent_uuid: "8ee7401f0286413092bba932ce0a7ba2" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 44743 } } peers { permanent_uuid: "58a4886f5b7a4aff99ca6f4a635db238" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 46417 } }
I20260809 15:35:40.277375 5207 leader_election.cc:288] T 3e2f2d410fc44708a2bb08bf05ef8ece P 6383b28fd60f46e6b9627721662f1918 [CANDIDATE]: Term 1 election: Requested vote from peers 8ee7401f0286413092bba932ce0a7ba2 (127.3.89.194:44743), 58a4886f5b7a4aff99ca6f4a635db238 (127.3.89.195:46417)
I20260809 15:35:40.277654 4970 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3e2f2d410fc44708a2bb08bf05ef8ece" candidate_uuid: "6383b28fd60f46e6b9627721662f1918" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8ee7401f0286413092bba932ce0a7ba2"
I20260809 15:35:40.277663 5125 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3e2f2d410fc44708a2bb08bf05ef8ece" candidate_uuid: "6383b28fd60f46e6b9627721662f1918" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "58a4886f5b7a4aff99ca6f4a635db238"
I20260809 15:35:40.277784 4970 raft_consensus.cc:3037] T 3e2f2d410fc44708a2bb08bf05ef8ece P 8ee7401f0286413092bba932ce0a7ba2 [term 0 FOLLOWER]: Advancing to term 1
I20260809 15:35:40.277794 5125 raft_consensus.cc:3037] T 3e2f2d410fc44708a2bb08bf05ef8ece P 58a4886f5b7a4aff99ca6f4a635db238 [term 0 FOLLOWER]: Advancing to term 1
I20260809 15:35:40.278896 5125 raft_consensus.cc:2454] T 3e2f2d410fc44708a2bb08bf05ef8ece P 58a4886f5b7a4aff99ca6f4a635db238 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6383b28fd60f46e6b9627721662f1918 in term 1.
I20260809 15:35:40.278896 4970 raft_consensus.cc:2454] T 3e2f2d410fc44708a2bb08bf05ef8ece P 8ee7401f0286413092bba932ce0a7ba2 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6383b28fd60f46e6b9627721662f1918 in term 1.
I20260809 15:35:40.279186 4741 leader_election.cc:302] T 3e2f2d410fc44708a2bb08bf05ef8ece P 6383b28fd60f46e6b9627721662f1918 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 3 yes votes; 0 no votes. yes voters: 58a4886f5b7a4aff99ca6f4a635db238, 6383b28fd60f46e6b9627721662f1918, 8ee7401f0286413092bba932ce0a7ba2; no voters:
I20260809 15:35:40.279333 5207 raft_consensus.cc:2781] T 3e2f2d410fc44708a2bb08bf05ef8ece P 6383b28fd60f46e6b9627721662f1918 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 15:35:40.279577 5207 raft_consensus.cc:686] T 3e2f2d410fc44708a2bb08bf05ef8ece P 6383b28fd60f46e6b9627721662f1918 [term 1 LEADER]: Becoming Leader. State: Replica: 6383b28fd60f46e6b9627721662f1918, State: Running, Role: LEADER
I20260809 15:35:40.279829 5207 consensus_queue.cc:237] T 3e2f2d410fc44708a2bb08bf05ef8ece P 6383b28fd60f46e6b9627721662f1918 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6383b28fd60f46e6b9627721662f1918" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 44379 } } peers { permanent_uuid: "8ee7401f0286413092bba932ce0a7ba2" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 44743 } } peers { permanent_uuid: "58a4886f5b7a4aff99ca6f4a635db238" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 46417 } }
I20260809 15:35:40.281291 5246 tablet_replica.cc:439] T 3e2f2d410fc44708a2bb08bf05ef8ece P 6383b28fd60f46e6b9627721662f1918: TxnStatusTablet state changed. Reason: New leader 6383b28fd60f46e6b9627721662f1918. Latest consensus state: current_term: 1 leader_uuid: "6383b28fd60f46e6b9627721662f1918" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6383b28fd60f46e6b9627721662f1918" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 44379 } } peers { permanent_uuid: "8ee7401f0286413092bba932ce0a7ba2" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 44743 } } peers { permanent_uuid: "58a4886f5b7a4aff99ca6f4a635db238" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 46417 } } }
I20260809 15:35:40.281420 5246 tablet_replica.cc:442] T 3e2f2d410fc44708a2bb08bf05ef8ece P 6383b28fd60f46e6b9627721662f1918: This TxnStatusTablet replica's current role is: LEADER
I20260809 15:35:40.281751 5248 txn_status_manager.cc:873] Waiting until node catch up with all replicated operations in previous term...
I20260809 15:35:40.281934 5248 txn_status_manager.cc:929] Loading transaction status metadata into memory...
I20260809 15:35:40.281944 4649 catalog_manager.cc:5495] T 3e2f2d410fc44708a2bb08bf05ef8ece P 6383b28fd60f46e6b9627721662f1918 reported cstate change: term changed from 0 to 1, leader changed from <none> to 6383b28fd60f46e6b9627721662f1918 (127.3.89.193). New cstate: current_term: 1 leader_uuid: "6383b28fd60f46e6b9627721662f1918" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6383b28fd60f46e6b9627721662f1918" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 44379 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "8ee7401f0286413092bba932ce0a7ba2" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 44743 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "58a4886f5b7a4aff99ca6f4a635db238" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 46417 } health_report { overall_health: UNKNOWN } } }
I20260809 15:35:40.303175 5194 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:35:40.299448 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45934 (local address 127.3.89.193:44379)
0809 15:35:40.299602 (+ 154us) server_negotiation.cc:206] Beginning negotiation
0809 15:35:40.299609 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 15:35:40.299624 (+ 15us) server_negotiation.cc:407] Connection header received
0809 15:35:40.299828 (+ 204us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 15:35:40.299832 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 15:35:40.299891 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 15:35:40.300012 (+ 121us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 15:35:40.300266 (+ 254us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:35:40.300890 (+ 624us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:35:40.301735 (+ 845us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:35:40.301963 (+ 228us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:35:40.302076 (+ 113us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:35:40.302373 (+ 297us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 15:35:40.302763 (+ 390us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 15:35:40.302889 (+ 126us) server_negotiation.cc:1035] Waiting for connection context
0809 15:35:40.302986 (+ 97us) server_negotiation.cc:299] Negotiation successful
0809 15:35:40.303037 (+ 51us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":51}
I20260809 15:35:40.303174 5240 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:35:40.299301 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.89.193:44379 (local address 127.0.0.1:45934)
0809 15:35:40.299553 (+ 252us) negotiation.cc:107] Waiting for socket to connect
0809 15:35:40.299570 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0809 15:35:40.299683 (+ 113us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 15:35:40.300026 (+ 343us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 15:35:40.300030 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 15:35:40.300039 (+ 9us) client_negotiation.cc:189] Negotiated authn=TOKEN
0809 15:35:40.300155 (+ 116us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:35:40.300161 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:35:40.301005 (+ 844us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:35:40.301010 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:35:40.301616 (+ 606us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:35:40.301623 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:35:40.302077 (+ 454us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:35:40.302081 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:35:40.302180 (+ 99us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:35:40.302226 (+ 46us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0809 15:35:40.302869 (+ 643us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0809 15:35:40.302887 (+ 18us) client_negotiation.cc:769] Sending connection context
0809 15:35:40.302975 (+ 88us) client_negotiation.cc:240] Negotiation successful
0809 15:35:40.303059 (+ 84us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":138}
I20260809 15:35:40.305908 4970 raft_consensus.cc:1261] T 3e2f2d410fc44708a2bb08bf05ef8ece P 8ee7401f0286413092bba932ce0a7ba2 [term 1 FOLLOWER]: Refusing update from remote peer 6383b28fd60f46e6b9627721662f1918: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 15:35:40.306102 5125 raft_consensus.cc:1261] T 3e2f2d410fc44708a2bb08bf05ef8ece P 58a4886f5b7a4aff99ca6f4a635db238 [term 1 FOLLOWER]: Refusing update from remote peer 6383b28fd60f46e6b9627721662f1918: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 15:35:40.306391 5207 consensus_queue.cc:1035] T 3e2f2d410fc44708a2bb08bf05ef8ece P 6383b28fd60f46e6b9627721662f1918 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8ee7401f0286413092bba932ce0a7ba2" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 44743 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 15:35:40.306617 5246 consensus_queue.cc:1035] T 3e2f2d410fc44708a2bb08bf05ef8ece P 6383b28fd60f46e6b9627721662f1918 [LEADER]: Connected to new peer: Peer: permanent_uuid: "58a4886f5b7a4aff99ca6f4a635db238" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 46417 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 15:35:40.308063 5232 tablet_replica.cc:439] T 3e2f2d410fc44708a2bb08bf05ef8ece P 8ee7401f0286413092bba932ce0a7ba2: TxnStatusTablet state changed. Reason: New leader 6383b28fd60f46e6b9627721662f1918. Latest consensus state: current_term: 1 leader_uuid: "6383b28fd60f46e6b9627721662f1918" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6383b28fd60f46e6b9627721662f1918" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 44379 } } peers { permanent_uuid: "8ee7401f0286413092bba932ce0a7ba2" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 44743 } } peers { permanent_uuid: "58a4886f5b7a4aff99ca6f4a635db238" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 46417 } } }
I20260809 15:35:40.308172 5232 tablet_replica.cc:442] T 3e2f2d410fc44708a2bb08bf05ef8ece P 8ee7401f0286413092bba932ce0a7ba2: This TxnStatusTablet replica's current role is: FOLLOWER
I20260809 15:35:40.309020 5220 tablet_replica.cc:439] T 3e2f2d410fc44708a2bb08bf05ef8ece P 58a4886f5b7a4aff99ca6f4a635db238: TxnStatusTablet state changed. Reason: New leader 6383b28fd60f46e6b9627721662f1918. Latest consensus state: current_term: 1 leader_uuid: "6383b28fd60f46e6b9627721662f1918" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6383b28fd60f46e6b9627721662f1918" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 44379 } } peers { permanent_uuid: "8ee7401f0286413092bba932ce0a7ba2" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 44743 } } peers { permanent_uuid: "58a4886f5b7a4aff99ca6f4a635db238" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 46417 } } }
I20260809 15:35:40.309134 5220 tablet_replica.cc:442] T 3e2f2d410fc44708a2bb08bf05ef8ece P 58a4886f5b7a4aff99ca6f4a635db238: This TxnStatusTablet replica's current role is: FOLLOWER
I20260809 15:35:40.309733 5246 tablet_replica.cc:439] T 3e2f2d410fc44708a2bb08bf05ef8ece P 6383b28fd60f46e6b9627721662f1918: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "6383b28fd60f46e6b9627721662f1918" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6383b28fd60f46e6b9627721662f1918" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 44379 } } peers { permanent_uuid: "8ee7401f0286413092bba932ce0a7ba2" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 44743 } } peers { permanent_uuid: "58a4886f5b7a4aff99ca6f4a635db238" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 46417 } } }
I20260809 15:35:40.309835 5246 tablet_replica.cc:442] T 3e2f2d410fc44708a2bb08bf05ef8ece P 6383b28fd60f46e6b9627721662f1918: This TxnStatusTablet replica's current role is: LEADER
I20260809 15:35:40.311308 5207 tablet_replica.cc:439] T 3e2f2d410fc44708a2bb08bf05ef8ece P 6383b28fd60f46e6b9627721662f1918: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "6383b28fd60f46e6b9627721662f1918" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6383b28fd60f46e6b9627721662f1918" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 44379 } } peers { permanent_uuid: "8ee7401f0286413092bba932ce0a7ba2" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 44743 } } peers { permanent_uuid: "58a4886f5b7a4aff99ca6f4a635db238" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 46417 } } }
I20260809 15:35:40.311450 5207 tablet_replica.cc:442] T 3e2f2d410fc44708a2bb08bf05ef8ece P 6383b28fd60f46e6b9627721662f1918: This TxnStatusTablet replica's current role is: LEADER
I20260809 15:35:40.312141 5213 mvcc.cc:205] Tried to move back new op lower bound from 7316642776288129024 to 7316642776188190720. Current Snapshot: MvccSnapshot[applied={T|T < 7316642776288129024}]
I20260809 15:35:40.312299 5220 tablet_replica.cc:439] T 3e2f2d410fc44708a2bb08bf05ef8ece P 58a4886f5b7a4aff99ca6f4a635db238: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "6383b28fd60f46e6b9627721662f1918" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6383b28fd60f46e6b9627721662f1918" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 44379 } } peers { permanent_uuid: "8ee7401f0286413092bba932ce0a7ba2" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 44743 } } peers { permanent_uuid: "58a4886f5b7a4aff99ca6f4a635db238" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 46417 } } }
I20260809 15:35:40.312399 5220 tablet_replica.cc:442] T 3e2f2d410fc44708a2bb08bf05ef8ece P 58a4886f5b7a4aff99ca6f4a635db238: This TxnStatusTablet replica's current role is: FOLLOWER
I20260809 15:35:40.313060 5232 tablet_replica.cc:439] T 3e2f2d410fc44708a2bb08bf05ef8ece P 8ee7401f0286413092bba932ce0a7ba2: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "6383b28fd60f46e6b9627721662f1918" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6383b28fd60f46e6b9627721662f1918" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 44379 } } peers { permanent_uuid: "8ee7401f0286413092bba932ce0a7ba2" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 44743 } } peers { permanent_uuid: "58a4886f5b7a4aff99ca6f4a635db238" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 46417 } } }
I20260809 15:35:40.313170 5232 tablet_replica.cc:442] T 3e2f2d410fc44708a2bb08bf05ef8ece P 8ee7401f0286413092bba932ce0a7ba2: This TxnStatusTablet replica's current role is: FOLLOWER
I20260809 15:35:40.320317 5198 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:35:40.316522 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48434 (local address 127.3.89.194:44743)
0809 15:35:40.316645 (+ 123us) server_negotiation.cc:206] Beginning negotiation
0809 15:35:40.316650 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 15:35:40.316664 (+ 14us) server_negotiation.cc:407] Connection header received
0809 15:35:40.316784 (+ 120us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 15:35:40.316787 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 15:35:40.316841 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 15:35:40.316944 (+ 103us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 15:35:40.317148 (+ 204us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:35:40.317907 (+ 759us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:35:40.318690 (+ 783us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:35:40.319078 (+ 388us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:35:40.319181 (+ 103us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:35:40.319432 (+ 251us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 15:35:40.319907 (+ 475us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 15:35:40.320019 (+ 112us) server_negotiation.cc:1035] Waiting for connection context
0809 15:35:40.320126 (+ 107us) server_negotiation.cc:299] Negotiation successful
0809 15:35:40.320175 (+ 49us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":55}
I20260809 15:35:40.331496 5255 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:35:40.327886 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.89.193:44379 (local address 127.3.89.194:55463)
0809 15:35:40.328254 (+ 368us) negotiation.cc:107] Waiting for socket to connect
0809 15:35:40.328266 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0809 15:35:40.328382 (+ 116us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 15:35:40.328608 (+ 226us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 15:35:40.328611 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 15:35:40.328619 (+ 8us) client_negotiation.cc:189] Negotiated authn=TOKEN
0809 15:35:40.328732 (+ 113us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:35:40.328745 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:35:40.329460 (+ 715us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:35:40.329466 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:35:40.330213 (+ 747us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:35:40.330222 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:35:40.330690 (+ 468us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:35:40.330694 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:35:40.330812 (+ 118us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:35:40.330865 (+ 53us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0809 15:35:40.331214 (+ 349us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0809 15:35:40.331221 (+ 7us) client_negotiation.cc:769] Sending connection context
0809 15:35:40.331281 (+ 60us) client_negotiation.cc:240] Negotiation successful
0809 15:35:40.331353 (+ 72us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":278,"thread_start_us":121,"threads_started":1}
I20260809 15:35:40.331480 5194 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:35:40.327979 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.89.194:55463 (local address 127.3.89.193:44379)
0809 15:35:40.328116 (+ 137us) server_negotiation.cc:206] Beginning negotiation
0809 15:35:40.328123 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 15:35:40.328338 (+ 215us) server_negotiation.cc:407] Connection header received
0809 15:35:40.328467 (+ 129us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 15:35:40.328470 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 15:35:40.328517 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 15:35:40.328600 (+ 83us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 15:35:40.328841 (+ 241us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:35:40.329334 (+ 493us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:35:40.330344 (+ 1010us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:35:40.330559 (+ 215us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:35:40.330674 (+ 115us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:35:40.331049 (+ 375us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 15:35:40.331118 (+ 69us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 15:35:40.331198 (+ 80us) server_negotiation.cc:1035] Waiting for connection context
0809 15:35:40.331297 (+ 99us) server_negotiation.cc:299] Negotiation successful
0809 15:35:40.331345 (+ 48us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":55}
I20260809 15:35:40.338346 5125 raft_consensus.cc:1261] T f0554ad5e76b4f1a83ff77194fd3be39 P 58a4886f5b7a4aff99ca6f4a635db238 [term 1 FOLLOWER]: Refusing update from remote peer 8ee7401f0286413092bba932ce0a7ba2: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 15:35:40.338371 4816 raft_consensus.cc:1261] T f0554ad5e76b4f1a83ff77194fd3be39 P 6383b28fd60f46e6b9627721662f1918 [term 1 FOLLOWER]: Refusing update from remote peer 8ee7401f0286413092bba932ce0a7ba2: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 15:35:40.338820 5232 consensus_queue.cc:1035] T f0554ad5e76b4f1a83ff77194fd3be39 P 8ee7401f0286413092bba932ce0a7ba2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6383b28fd60f46e6b9627721662f1918" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 44379 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 15:35:40.339011 5205 consensus_queue.cc:1035] T f0554ad5e76b4f1a83ff77194fd3be39 P 8ee7401f0286413092bba932ce0a7ba2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "58a4886f5b7a4aff99ca6f4a635db238" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 46417 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 15:35:40.362063 5261 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:35:40.357822 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.89.194:44743 (local address 127.3.89.193:49737)
0809 15:35:40.358163 (+ 341us) negotiation.cc:107] Waiting for socket to connect
0809 15:35:40.358177 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0809 15:35:40.358266 (+ 89us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 15:35:40.358510 (+ 244us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 15:35:40.358512 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 15:35:40.358517 (+ 5us) client_negotiation.cc:189] Negotiated authn=TOKEN
0809 15:35:40.358622 (+ 105us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:35:40.358626 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:35:40.359818 (+ 1192us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:35:40.359820 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:35:40.360719 (+ 899us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:35:40.360729 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:35:40.361313 (+ 584us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:35:40.361315 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:35:40.361384 (+ 69us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:35:40.361425 (+ 41us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0809 15:35:40.361756 (+ 331us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0809 15:35:40.361762 (+ 6us) client_negotiation.cc:769] Sending connection context
0809 15:35:40.361846 (+ 84us) client_negotiation.cc:240] Negotiation successful
0809 15:35:40.361924 (+ 78us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":188,"thread_start_us":118,"threads_started":1}
I20260809 15:35:40.362047 5198 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:35:40.357868 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.89.193:49737 (local address 127.3.89.194:44743)
0809 15:35:40.357983 (+ 115us) server_negotiation.cc:206] Beginning negotiation
0809 15:35:40.357987 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 15:35:40.358249 (+ 262us) server_negotiation.cc:407] Connection header received
0809 15:35:40.358360 (+ 111us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 15:35:40.358363 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 15:35:40.358424 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 15:35:40.358509 (+ 85us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 15:35:40.358709 (+ 200us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:35:40.359685 (+ 976us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:35:40.360842 (+ 1157us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:35:40.361186 (+ 344us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:35:40.361287 (+ 101us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:35:40.361557 (+ 270us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 15:35:40.361666 (+ 109us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 15:35:40.361732 (+ 66us) server_negotiation.cc:1035] Waiting for connection context
0809 15:35:40.361859 (+ 127us) server_negotiation.cc:299] Negotiation successful
0809 15:35:40.361909 (+ 50us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":44}
I20260809 15:35:40.388190 4651 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:46748:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 15:35:40.388348 4651 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:46748:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 15:35:40.390534 4651 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 2d52ec43c6d145ccbb52cd484ca05989: Sending DeleteTablet for 3 replicas of tablet f0554ad5e76b4f1a83ff77194fd3be39
I20260809 15:35:40.391371 5105 tablet_service.cc:1514] Processing DeleteTablet for tablet f0554ad5e76b4f1a83ff77194fd3be39 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 15:35:40 UTC) from {username='kudu', principal='kudu/127.3.89.254@KRBTEST.COM'} at 127.0.0.1:48486
I20260809 15:35:40.391506 4948 tablet_service.cc:1514] Processing DeleteTablet for tablet f0554ad5e76b4f1a83ff77194fd3be39 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 15:35:40 UTC) from {username='kudu', principal='kudu/127.3.89.254@KRBTEST.COM'} at 127.0.0.1:48422
I20260809 15:35:40.391708 5225 tablet_replica.cc:330] T f0554ad5e76b4f1a83ff77194fd3be39 P 58a4886f5b7a4aff99ca6f4a635db238: stopping tablet replica
I20260809 15:35:40.391722 5226 tablet_replica.cc:330] T f0554ad5e76b4f1a83ff77194fd3be39 P 8ee7401f0286413092bba932ce0a7ba2: stopping tablet replica
I20260809 15:35:40.391834 5226 raft_consensus.cc:2229] T f0554ad5e76b4f1a83ff77194fd3be39 P 8ee7401f0286413092bba932ce0a7ba2 [term 1 LEADER]: Raft consensus shutting down.
I20260809 15:35:40.391835 5225 raft_consensus.cc:2229] T f0554ad5e76b4f1a83ff77194fd3be39 P 58a4886f5b7a4aff99ca6f4a635db238 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 15:35:40.391950 5225 raft_consensus.cc:2258] T f0554ad5e76b4f1a83ff77194fd3be39 P 58a4886f5b7a4aff99ca6f4a635db238 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 15:35:40.392082 5226 raft_consensus.cc:2258] T f0554ad5e76b4f1a83ff77194fd3be39 P 8ee7401f0286413092bba932ce0a7ba2 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 15:35:40.392824 5226 ts_tablet_manager.cc:1905] T f0554ad5e76b4f1a83ff77194fd3be39 P 8ee7401f0286413092bba932ce0a7ba2: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 15:35:40.393154 5225 ts_tablet_manager.cc:1905] T f0554ad5e76b4f1a83ff77194fd3be39 P 58a4886f5b7a4aff99ca6f4a635db238: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 15:35:40.395721 5226 ts_tablet_manager.cc:1918] T f0554ad5e76b4f1a83ff77194fd3be39 P 8ee7401f0286413092bba932ce0a7ba2: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260809 15:35:40.395792 5226 log.cc:1196] T f0554ad5e76b4f1a83ff77194fd3be39 P 8ee7401f0286413092bba932ce0a7ba2: Deleting WAL directory at /tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786289732823074-3431-0/minicluster-data/ts-1/wal/wals/f0554ad5e76b4f1a83ff77194fd3be39
I20260809 15:35:40.395993 5225 ts_tablet_manager.cc:1918] T f0554ad5e76b4f1a83ff77194fd3be39 P 58a4886f5b7a4aff99ca6f4a635db238: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260809 15:35:40.396044 5226 ts_tablet_manager.cc:1939] T f0554ad5e76b4f1a83ff77194fd3be39 P 8ee7401f0286413092bba932ce0a7ba2: Deleting consensus metadata
I20260809 15:35:40.396064 5225 log.cc:1196] T f0554ad5e76b4f1a83ff77194fd3be39 P 58a4886f5b7a4aff99ca6f4a635db238: Deleting WAL directory at /tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786289732823074-3431-0/minicluster-data/ts-2/wal/wals/f0554ad5e76b4f1a83ff77194fd3be39
I20260809 15:35:40.396339 5225 ts_tablet_manager.cc:1939] T f0554ad5e76b4f1a83ff77194fd3be39 P 58a4886f5b7a4aff99ca6f4a635db238: Deleting consensus metadata
I20260809 15:35:40.397080 4633 catalog_manager.cc:4841] TS 8ee7401f0286413092bba932ce0a7ba2 (127.3.89.194:44743): tablet f0554ad5e76b4f1a83ff77194fd3be39 (table test-table [id=843e9a7bd5c640d4b13021f9fd530640]) successfully deleted
I20260809 15:35:40.397275 4632 catalog_manager.cc:4841] TS 58a4886f5b7a4aff99ca6f4a635db238 (127.3.89.195:46417): tablet f0554ad5e76b4f1a83ff77194fd3be39 (table test-table [id=843e9a7bd5c640d4b13021f9fd530640]) successfully deleted
I20260809 15:35:40.397792 4796 tablet_service.cc:1514] Processing DeleteTablet for tablet f0554ad5e76b4f1a83ff77194fd3be39 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 15:35:40 UTC) from {username='kudu', principal='kudu/127.3.89.254@KRBTEST.COM'} at 127.0.0.1:45926
I20260809 15:35:40.398010 5224 tablet_replica.cc:330] T f0554ad5e76b4f1a83ff77194fd3be39 P 6383b28fd60f46e6b9627721662f1918: stopping tablet replica
I20260809 15:35:40.398123 5224 raft_consensus.cc:2229] T f0554ad5e76b4f1a83ff77194fd3be39 P 6383b28fd60f46e6b9627721662f1918 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 15:35:40.398229 5224 raft_consensus.cc:2258] T f0554ad5e76b4f1a83ff77194fd3be39 P 6383b28fd60f46e6b9627721662f1918 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 15:35:40.399609 5224 ts_tablet_manager.cc:1905] T f0554ad5e76b4f1a83ff77194fd3be39 P 6383b28fd60f46e6b9627721662f1918: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 15:35:40.402295 5224 ts_tablet_manager.cc:1918] T f0554ad5e76b4f1a83ff77194fd3be39 P 6383b28fd60f46e6b9627721662f1918: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260809 15:35:40.402367 5224 log.cc:1196] T f0554ad5e76b4f1a83ff77194fd3be39 P 6383b28fd60f46e6b9627721662f1918: Deleting WAL directory at /tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786289732823074-3431-0/minicluster-data/ts-0/wal/wals/f0554ad5e76b4f1a83ff77194fd3be39
I20260809 15:35:40.402658 5224 ts_tablet_manager.cc:1939] T f0554ad5e76b4f1a83ff77194fd3be39 P 6383b28fd60f46e6b9627721662f1918: Deleting consensus metadata
I20260809 15:35:40.403420 4635 catalog_manager.cc:4841] TS 6383b28fd60f46e6b9627721662f1918 (127.3.89.193:44379): tablet f0554ad5e76b4f1a83ff77194fd3be39 (table test-table [id=843e9a7bd5c640d4b13021f9fd530640]) successfully deleted
Aug 09 15:35:40 dist-test-slave-2w3w krb5kdc[4599](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786289739, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.3.89.193@KRBTEST.COM
I20260809 15:35:40.414528 5194 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:35:40.399791 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45944 (local address 127.3.89.193:44379)
0809 15:35:40.399921 (+ 130us) server_negotiation.cc:206] Beginning negotiation
0809 15:35:40.399925 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 15:35:40.400681 (+ 756us) server_negotiation.cc:407] Connection header received
0809 15:35:40.400917 (+ 236us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 15:35:40.400921 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 15:35:40.400977 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 15:35:40.401034 (+ 57us) server_negotiation.cc:226] Negotiated authn=SASL
0809 15:35:40.402139 (+ 1105us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:35:40.403079 (+ 940us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:35:40.404441 (+ 1362us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:35:40.404793 (+ 352us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:35:40.404908 (+ 115us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:35:40.409714 (+ 4806us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 15:35:40.409725 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 15:35:40.409741 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 15:35:40.409771 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0809 15:35:40.411924 (+ 2153us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 15:35:40.412431 (+ 507us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 15:35:40.412438 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 15:35:40.412440 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 15:35:40.412527 (+ 87us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 15:35:40.412827 (+ 300us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 15:35:40.412830 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 15:35:40.412831 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 15:35:40.413141 (+ 310us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 15:35:40.413310 (+ 169us) server_negotiation.cc:1035] Waiting for connection context
0809 15:35:40.413717 (+ 407us) server_negotiation.cc:299] Negotiation successful
0809 15:35:40.413845 (+ 128us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":41}
W20260809 15:35:40.415725 4858 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.SetFlag from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:45944
I20260809 15:35:40.427430 4701 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:35:40.417588 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46776 (local address 127.3.89.254:45563)
0809 15:35:40.417713 (+ 125us) server_negotiation.cc:206] Beginning negotiation
0809 15:35:40.417718 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 15:35:40.417845 (+ 127us) server_negotiation.cc:407] Connection header received
0809 15:35:40.417961 (+ 116us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 15:35:40.417963 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 15:35:40.418022 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 15:35:40.418110 (+ 88us) server_negotiation.cc:226] Negotiated authn=SASL
0809 15:35:40.418939 (+ 829us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:35:40.419783 (+ 844us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:35:40.421007 (+ 1224us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:35:40.421375 (+ 368us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:35:40.421498 (+ 123us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:35:40.423317 (+ 1819us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 15:35:40.423322 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 15:35:40.423325 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 15:35:40.423355 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0809 15:35:40.425251 (+ 1896us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 15:35:40.425805 (+ 554us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 15:35:40.425808 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 15:35:40.425811 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 15:35:40.425909 (+ 98us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 15:35:40.426232 (+ 323us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 15:35:40.426235 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 15:35:40.426238 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 15:35:40.426491 (+ 253us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 15:35:40.426630 (+ 139us) server_negotiation.cc:1035] Waiting for connection context
0809 15:35:40.426901 (+ 271us) server_negotiation.cc:299] Negotiation successful
0809 15:35:40.427102 (+ 201us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":51}
W20260809 15:35:40.427837 4651 server_base.cc:1116] Unauthorized access attempt to method kudu.master.MasterService.TSHeartbeat from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:46776
I20260809 15:35:40.429487 3431 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskm314YR/build/debug/bin/kudu with pid 4725
I20260809 15:35:40.439797 3431 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskm314YR/build/debug/bin/kudu with pid 4876
I20260809 15:35:40.448045 3431 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskm314YR/build/debug/bin/kudu with pid 5031
I20260809 15:35:40.455812 3431 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskm314YR/build/debug/bin/kudu with pid 4616
2026-08-09T15:35:40Z chronyd exiting
[ OK ] SecurityITest.SmokeTestAsAuthorizedUser (2687 ms)
[ RUN ] SecurityITest.TxnSmokeWithDifferentUserTypes
Loading random data
Initializing database '/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786289732823074-3431-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 15:35:40 dist-test-slave-2w3w krb5kdc[5270](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 15:35:40 dist-test-slave-2w3w krb5kdc[5270](info): set up 2 sockets
Aug 09 15:35:40 dist-test-slave-2w3w krb5kdc[5270](info): commencing operation
krb5kdc: starting...
W20260809 15:35:41.524055 3431 mini_kdc.cc:121] Time spent starting KDC: real 1.044s user 0.005s sys 0.002s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786289732823074-3431-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786289732823074-3431-0/krb5kdc/test-user.keytab.
Aug 09 15:35:41 dist-test-slave-2w3w krb5kdc[5270](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786289741, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T15:35:41Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T15:35:41Z Disabled control of system clock
WARNING: no policy specified for kudu/127.3.89.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.3.89.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.3.89.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786289732823074-3431-0/krb5kdc/kudu_127.3.89.254.keytab.
Entry for principal kudu/127.3.89.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786289732823074-3431-0/krb5kdc/kudu_127.3.89.254.keytab.
I20260809 15:35:41.647233 3431 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskm314YR/build/debug/bin/kudu
/tmp/dist-test-taskm314YR/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786289732823074-3431-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786289732823074-3431-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786289732823074-3431-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786289732823074-3431-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.3.89.254:33125
--webserver_interface=127.3.89.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786289732823074-3431-0/krb5kdc/kudu_127.3.89.254.keytab
--principal=kudu/127.3.89.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.3.89.212:42515
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.3.89.254:33125
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--txn_manager_enabled=true with env {KRB5CCNAME=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786289732823074-3431-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786289732823074-3431-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786289732823074-3431-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 15:35:41.758711 5284 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 15:35:41.759006 5284 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 15:35:41.759069 5284 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 15:35:41.763060 5284 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 15:35:41.763109 5284 flags.cc:425] Enabled experimental flag: --txn_manager_enabled=true
W20260809 15:35:41.763126 5284 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 15:35:41.763144 5284 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 15:35:41.763160 5284 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 15:35:41.763175 5284 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 15:35:41.768630 5284 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.3.89.212:42515
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786289732823074-3431-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786289732823074-3431-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.3.89.254:33125
--txn_manager_enabled=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786289732823074-3431-0/krb5kdc/kudu_127.3.89.254.keytab
--principal=kudu/127.3.89.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.3.89.254:33125
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786289732823074-3431-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.3.89.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.5284
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786289732823074-3431-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 09 Aug 2026 14:46:55 UTC on ccycloud.quasar-yswkmf.root.comops.site
build id 4576540
I20260809 15:35:41.769577 5284 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 15:35:41.770503 5284 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 15:35:41.779177 5293 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 15:35:41.779263 5284 server_base.cc:1034] running on GCE node
W20260809 15:35:41.779211 5291 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 15:35:41.779211 5290 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 15:35:41.779992 5284 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 15:35:41.780982 5284 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 15:35:41.782138 5284 hybrid_clock.cc:648] HybridClock initialized: now 1786289741782113 us; error 40 us; skew 500 ppm
Aug 09 15:35:41 dist-test-slave-2w3w krb5kdc[5270](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786289741, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.89.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 15:35:41.784981 5284 init.cc:377] Logged in from keytab as kudu/127.3.89.254@KRBTEST.COM (short username kudu)
I20260809 15:35:41.785782 5284 webserver.cc:460] Webserver started at http://127.3.89.254:41551/ using document root <none> and password file <none>
I20260809 15:35:41.786365 5284 fs_manager.cc:359] Metadata directory not provided
I20260809 15:35:41.786448 5284 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 15:35:41.786610 5284 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 15:35:41.788635 5284 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786289732823074-3431-0/minicluster-data/master-0/data/instance:
uuid: "726056f89ec1474594dd4b44dcfadbfe"
format_stamp: "Formatted at 2026-08-09 15:35:41 on dist-test-slave-2w3w"
server_key: "6a7369418271ad5109f659d3f5738835"
server_key_iv: "697ec94731c4586d6960bb81b2fc857e"
server_key_version: "encryptionkey@0"
I20260809 15:35:41.789150 5284 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786289732823074-3431-0/minicluster-data/master-0/wal/instance:
uuid: "726056f89ec1474594dd4b44dcfadbfe"
format_stamp: "Formatted at 2026-08-09 15:35:41 on dist-test-slave-2w3w"
server_key: "6a7369418271ad5109f659d3f5738835"
server_key_iv: "697ec94731c4586d6960bb81b2fc857e"
server_key_version: "encryptionkey@0"
I20260809 15:35:41.792742 5284 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.002s sys 0.003s
I20260809 15:35:41.795137 5300 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 15:35:41.796334 5284 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.001s
I20260809 15:35:41.796442 5284 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786289732823074-3431-0/minicluster-data/master-0/data,/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786289732823074-3431-0/minicluster-data/master-0/wal
uuid: "726056f89ec1474594dd4b44dcfadbfe"
format_stamp: "Formatted at 2026-08-09 15:35:41 on dist-test-slave-2w3w"
server_key: "6a7369418271ad5109f659d3f5738835"
server_key_iv: "697ec94731c4586d6960bb81b2fc857e"
server_key_version: "encryptionkey@0"
I20260809 15:35:41.796530 5284 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786289732823074-3431-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786289732823074-3431-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786289732823074-3431-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 15:35:41.812597 5284 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 15:35:41.815855 5284 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 15:35:41.816020 5284 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 15:35:41.823930 5284 rpc_server.cc:307] RPC server started. Bound to: 127.3.89.254:33125
I20260809 15:35:41.823956 5365 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.3.89.254:33125 every 8 connection(s)
I20260809 15:35:41.824867 5284 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786289732823074-3431-0/minicluster-data/master-0/data/info.pb
I20260809 15:35:41.827525 5366 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 15:35:41.832703 5366 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 726056f89ec1474594dd4b44dcfadbfe: Bootstrap starting.
I20260809 15:35:41.833442 3431 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskm314YR/build/debug/bin/kudu as pid 5284
I20260809 15:35:41.833556 3431 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786289732823074-3431-0/minicluster-data/master-0/wal/instance
I20260809 15:35:41.833837 3431 external_mini_cluster.cc:1428] Setting key 4059436ba85b877b23dc73f9df59a21f
I20260809 15:35:41.835532 5366 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 726056f89ec1474594dd4b44dcfadbfe: Neither blocks nor log segments found. Creating new log.
I20260809 15:35:41.836509 5366 log.cc:824] T 00000000000000000000000000000000 P 726056f89ec1474594dd4b44dcfadbfe: Log is configured to *not* fsync() on all Append() calls
I20260809 15:35:41.838375 5366 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 726056f89ec1474594dd4b44dcfadbfe: No bootstrap required, opened a new log
I20260809 15:35:41.842097 5366 raft_consensus.cc:348] T 00000000000000000000000000000000 P 726056f89ec1474594dd4b44dcfadbfe [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "726056f89ec1474594dd4b44dcfadbfe" member_type: VOTER last_known_addr { host: "127.3.89.254" port: 33125 } }
I20260809 15:35:41.842303 5366 raft_consensus.cc:374] T 00000000000000000000000000000000 P 726056f89ec1474594dd4b44dcfadbfe [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 15:35:41.842427 5366 raft_consensus.cc:729] T 00000000000000000000000000000000 P 726056f89ec1474594dd4b44dcfadbfe [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 726056f89ec1474594dd4b44dcfadbfe, State: Initialized, Role: FOLLOWER
Aug 09 15:35:41 dist-test-slave-2w3w krb5kdc[5270](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786289741, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.3.89.254@KRBTEST.COM
I20260809 15:35:41.842934 5366 consensus_queue.cc:260] T 00000000000000000000000000000000 P 726056f89ec1474594dd4b44dcfadbfe [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "726056f89ec1474594dd4b44dcfadbfe" member_type: VOTER last_known_addr { host: "127.3.89.254" port: 33125 } }
I20260809 15:35:41.843106 5366 raft_consensus.cc:388] T 00000000000000000000000000000000 P 726056f89ec1474594dd4b44dcfadbfe [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 15:35:41.843189 5366 raft_consensus.cc:482] T 00000000000000000000000000000000 P 726056f89ec1474594dd4b44dcfadbfe [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 15:35:41.843286 5366 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 726056f89ec1474594dd4b44dcfadbfe [term 0 FOLLOWER]: Advancing to term 1
I20260809 15:35:41.844302 5366 raft_consensus.cc:504] T 00000000000000000000000000000000 P 726056f89ec1474594dd4b44dcfadbfe [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "726056f89ec1474594dd4b44dcfadbfe" member_type: VOTER last_known_addr { host: "127.3.89.254" port: 33125 } }
I20260809 15:35:41.844671 5366 leader_election.cc:302] T 00000000000000000000000000000000 P 726056f89ec1474594dd4b44dcfadbfe [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 726056f89ec1474594dd4b44dcfadbfe; no voters:
I20260809 15:35:41.844993 5366 leader_election.cc:288] T 00000000000000000000000000000000 P 726056f89ec1474594dd4b44dcfadbfe [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 15:35:41.845108 5372 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 726056f89ec1474594dd4b44dcfadbfe [term 1 FOLLOWER]: Leader election won for term 1
I20260809 15:35:41.845317 5372 raft_consensus.cc:686] T 00000000000000000000000000000000 P 726056f89ec1474594dd4b44dcfadbfe [term 1 LEADER]: Becoming Leader. State: Replica: 726056f89ec1474594dd4b44dcfadbfe, State: Running, Role: LEADER
I20260809 15:35:41.845643 5372 consensus_queue.cc:237] T 00000000000000000000000000000000 P 726056f89ec1474594dd4b44dcfadbfe [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "726056f89ec1474594dd4b44dcfadbfe" member_type: VOTER last_known_addr { host: "127.3.89.254" port: 33125 } }
I20260809 15:35:41.846414 5366 sys_catalog.cc:564] T 00000000000000000000000000000000 P 726056f89ec1474594dd4b44dcfadbfe [sys.catalog]: configured and running, proceeding with master startup.
I20260809 15:35:41.847651 5374 sys_catalog.cc:455] T 00000000000000000000000000000000 P 726056f89ec1474594dd4b44dcfadbfe [sys.catalog]: SysCatalogTable state changed. Reason: New leader 726056f89ec1474594dd4b44dcfadbfe. Latest consensus state: current_term: 1 leader_uuid: "726056f89ec1474594dd4b44dcfadbfe" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "726056f89ec1474594dd4b44dcfadbfe" member_type: VOTER last_known_addr { host: "127.3.89.254" port: 33125 } } }
I20260809 15:35:41.847785 5374 sys_catalog.cc:458] T 00000000000000000000000000000000 P 726056f89ec1474594dd4b44dcfadbfe [sys.catalog]: This master's current role is: LEADER
I20260809 15:35:41.847662 5373 sys_catalog.cc:455] T 00000000000000000000000000000000 P 726056f89ec1474594dd4b44dcfadbfe [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "726056f89ec1474594dd4b44dcfadbfe" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "726056f89ec1474594dd4b44dcfadbfe" member_type: VOTER last_known_addr { host: "127.3.89.254" port: 33125 } } }
I20260809 15:35:41.847990 5373 sys_catalog.cc:458] T 00000000000000000000000000000000 P 726056f89ec1474594dd4b44dcfadbfe [sys.catalog]: This master's current role is: LEADER
I20260809 15:35:41.848286 5377 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 15:35:41.849097 5370 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:35:41.835046 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44994 (local address 127.3.89.254:33125)
0809 15:35:41.835476 (+ 430us) server_negotiation.cc:206] Beginning negotiation
0809 15:35:41.835491 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0809 15:35:41.835524 (+ 33us) server_negotiation.cc:407] Connection header received
0809 15:35:41.836505 (+ 981us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 15:35:41.836528 (+ 23us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 15:35:41.836983 (+ 455us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 15:35:41.837306 (+ 323us) server_negotiation.cc:226] Negotiated authn=SASL
0809 15:35:41.838129 (+ 823us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:35:41.839270 (+ 1141us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:35:41.840156 (+ 886us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:35:41.840631 (+ 475us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:35:41.840800 (+ 169us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:35:41.843571 (+ 2771us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 15:35:41.843576 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 15:35:41.843588 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 15:35:41.843614 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0809 15:35:41.845513 (+ 1899us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 15:35:41.846051 (+ 538us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 15:35:41.846057 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 15:35:41.846063 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0809 15:35:41.846205 (+ 142us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 15:35:41.846526 (+ 321us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 15:35:41.846529 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 15:35:41.846531 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 15:35:41.847159 (+ 628us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 15:35:41.847429 (+ 270us) server_negotiation.cc:1035] Waiting for connection context
0809 15:35:41.847719 (+ 290us) server_negotiation.cc:299] Negotiation successful
0809 15:35:41.848056 (+ 337us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":298,"thread_start_us":128,"threads_started":1}
I20260809 15:35:41.851903 5377 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 15:35:41.857259 5377 catalog_manager.cc:1347] Generated new cluster ID: 86ffee554fd24471b1c6331a5395f070
I20260809 15:35:41.857324 5377 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 15:35:41.875113 5377 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 15:35:41.876366 5377 catalog_manager.cc:1504] Loading token signing keys...
I20260809 15:35:41.884378 5377 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 726056f89ec1474594dd4b44dcfadbfe: Generated new TSK 0
I20260809 15:35:41.885169 5377 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.3.89.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.3.89.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.3.89.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786289732823074-3431-0/krb5kdc/kudu_127.3.89.193.keytab.
Entry for principal kudu/127.3.89.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786289732823074-3431-0/krb5kdc/kudu_127.3.89.193.keytab.
I20260809 15:35:41.918442 3431 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskm314YR/build/debug/bin/kudu
/tmp/dist-test-taskm314YR/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786289732823074-3431-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786289732823074-3431-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786289732823074-3431-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786289732823074-3431-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.3.89.193:0
--local_ip_for_outbound_sockets=127.3.89.193
--webserver_interface=127.3.89.193
--webserver_port=0
--tserver_master_addrs=127.3.89.254:33125
--keytab_file=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786289732823074-3431-0/krb5kdc/kudu_127.3.89.193.keytab
--principal=kudu/127.3.89.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.3.89.212:42515
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true
--txn_keepalive_interval_ms=500 with env {KRB5CCNAME=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786289732823074-3431-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786289732823074-3431-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786289732823074-3431-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 15:35:42.029807 5393 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 15:35:42.030050 5393 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 15:35:42.030094 5393 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 15:35:42.034053 5393 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 15:35:42.034106 5393 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 15:35:42.034173 5393 flags.cc:425] Enabled experimental flag: --txn_keepalive_interval_ms=500
W20260809 15:35:42.034202 5393 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260809 15:35:42.034267 5393 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.3.89.193
I20260809 15:35:42.039788 5393 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.3.89.212:42515
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786289732823074-3431-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786289732823074-3431-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786289732823074-3431-0/krb5kdc/kudu_127.3.89.193.keytab
--principal=kudu/127.3.89.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.3.89.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786289732823074-3431-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.3.89.193
--webserver_port=0
--txn_keepalive_interval_ms=500
--enable_txn_system_client_init=true
--tserver_master_addrs=127.3.89.254:33125
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.5393
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.3.89.193
--log_dir=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786289732823074-3431-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 09 Aug 2026 14:46:55 UTC on ccycloud.quasar-yswkmf.root.comops.site
build id 4576540
I20260809 15:35:42.040781 5393 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 15:35:42.041793 5393 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 15:35:42.051357 5401 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 15:35:42.051352 5398 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 15:35:42.051352 5399 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 15:35:42.051641 5393 server_base.cc:1034] running on GCE node
I20260809 15:35:42.052096 5393 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 15:35:42.052693 5393 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 15:35:42.053857 5393 hybrid_clock.cc:648] HybridClock initialized: now 1786289742053840 us; error 29 us; skew 500 ppm
Aug 09 15:35:42 dist-test-slave-2w3w krb5kdc[5270](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786289742, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.89.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 15:35:42.056762 5393 init.cc:377] Logged in from keytab as kudu/127.3.89.193@KRBTEST.COM (short username kudu)
I20260809 15:35:42.057597 5393 webserver.cc:460] Webserver started at http://127.3.89.193:36463/ using document root <none> and password file <none>
I20260809 15:35:42.058142 5393 fs_manager.cc:359] Metadata directory not provided
I20260809 15:35:42.058234 5393 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 15:35:42.058487 5393 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 15:35:42.060559 5393 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786289732823074-3431-0/minicluster-data/ts-0/data/instance:
uuid: "66c38532da714eceb1ef3488f4f5263d"
format_stamp: "Formatted at 2026-08-09 15:35:42 on dist-test-slave-2w3w"
server_key: "cddb62f6821bad6c20201d9bb0dc4504"
server_key_iv: "b755beb3bc41981977e5f93ab0c9772a"
server_key_version: "encryptionkey@0"
I20260809 15:35:42.061108 5393 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786289732823074-3431-0/minicluster-data/ts-0/wal/instance:
uuid: "66c38532da714eceb1ef3488f4f5263d"
format_stamp: "Formatted at 2026-08-09 15:35:42 on dist-test-slave-2w3w"
server_key: "cddb62f6821bad6c20201d9bb0dc4504"
server_key_iv: "b755beb3bc41981977e5f93ab0c9772a"
server_key_version: "encryptionkey@0"
I20260809 15:35:42.064745 5393 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.005s sys 0.000s
I20260809 15:35:42.067127 5408 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 15:35:42.068373 5393 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260809 15:35:42.068480 5393 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786289732823074-3431-0/minicluster-data/ts-0/data,/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786289732823074-3431-0/minicluster-data/ts-0/wal
uuid: "66c38532da714eceb1ef3488f4f5263d"
format_stamp: "Formatted at 2026-08-09 15:35:42 on dist-test-slave-2w3w"
server_key: "cddb62f6821bad6c20201d9bb0dc4504"
server_key_iv: "b755beb3bc41981977e5f93ab0c9772a"
server_key_version: "encryptionkey@0"
I20260809 15:35:42.068563 5393 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786289732823074-3431-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786289732823074-3431-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786289732823074-3431-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 15:35:42.091537 5393 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 15:35:42.094308 5393 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 15:35:42.094461 5393 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 15:35:42.096164 5393 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 15:35:42.096225 5393 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 15:35:42.096284 5393 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 15:35:42.096313 5393 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 09 15:35:42 dist-test-slave-2w3w krb5kdc[5270](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786289742, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.89.193@KRBTEST.COM for kudu/127.3.89.254@KRBTEST.COM
I20260809 15:35:42.107223 5393 rpc_server.cc:307] RPC server started. Bound to: 127.3.89.193:38695
I20260809 15:35:42.107313 5535 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.3.89.193:38695 every 8 connection(s)
I20260809 15:35:42.108322 5393 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786289732823074-3431-0/minicluster-data/ts-0/data/info.pb
I20260809 15:35:42.111068 5370 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:35:42.096848 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.89.193:34285 (local address 127.3.89.254:33125)
0809 15:35:42.097015 (+ 167us) server_negotiation.cc:206] Beginning negotiation
0809 15:35:42.097020 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 15:35:42.098241 (+ 1221us) server_negotiation.cc:407] Connection header received
0809 15:35:42.099523 (+ 1282us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 15:35:42.099527 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 15:35:42.099585 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 15:35:42.099690 (+ 105us) server_negotiation.cc:226] Negotiated authn=SASL
0809 15:35:42.101301 (+ 1611us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:35:42.101859 (+ 558us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:35:42.103623 (+ 1764us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:35:42.103992 (+ 369us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:35:42.104123 (+ 131us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:35:42.107159 (+ 3036us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 15:35:42.107164 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 15:35:42.107169 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 15:35:42.107197 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0809 15:35:42.108848 (+ 1651us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 15:35:42.109430 (+ 582us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 15:35:42.109434 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 15:35:42.109436 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 15:35:42.109532 (+ 96us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 15:35:42.109891 (+ 359us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 15:35:42.109894 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 15:35:42.109896 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 15:35:42.110085 (+ 189us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 15:35:42.110196 (+ 111us) server_negotiation.cc:1035] Waiting for connection context
0809 15:35:42.110674 (+ 478us) server_negotiation.cc:299] Negotiation successful
0809 15:35:42.110828 (+ 154us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":60}
I20260809 15:35:42.111728 5420 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:35:42.097622 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.89.254:33125 (local address 127.3.89.193:34285)
0809 15:35:42.098067 (+ 445us) negotiation.cc:107] Waiting for socket to connect
0809 15:35:42.098125 (+ 58us) client_negotiation.cc:174] Beginning negotiation
0809 15:35:42.099297 (+ 1172us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 15:35:42.099891 (+ 594us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 15:35:42.099900 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 15:35:42.100478 (+ 578us) client_negotiation.cc:189] Negotiated authn=SASL
0809 15:35:42.101150 (+ 672us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:35:42.101168 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:35:42.101989 (+ 821us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:35:42.101993 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:35:42.103479 (+ 1486us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:35:42.103491 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:35:42.104149 (+ 658us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:35:42.104152 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:35:42.104285 (+ 133us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:35:42.104848 (+ 563us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 15:35:42.104869 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0809 15:35:42.106882 (+ 2013us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 15:35:42.108986 (+ 2104us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 15:35:42.108992 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 15:35:42.109005 (+ 13us) client_negotiation.cc:760] calling sasl_client_step()
0809 15:35:42.109331 (+ 326us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 15:35:42.109640 (+ 309us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 15:35:42.109643 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 15:35:42.109645 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 15:35:42.109784 (+ 139us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 15:35:42.110197 (+ 413us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 15:35:42.110204 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 15:35:42.110433 (+ 229us) client_negotiation.cc:769] Sending connection context
0809 15:35:42.110610 (+ 177us) client_negotiation.cc:240] Negotiation successful
0809 15:35:42.110925 (+ 315us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":258,"thread_start_us":144,"threads_started":1}
I20260809 15:35:42.112919 5536 heartbeater.cc:347] Connected to a master server at 127.3.89.254:33125
I20260809 15:35:42.113058 5536 heartbeater.cc:464] Registering TS with master...
I20260809 15:35:42.113583 5536 heartbeater.cc:511] Master 127.3.89.254:33125 requested a full tablet report, sending...
I20260809 15:35:42.115183 5318 ts_manager.cc:194] Registered new tserver with Master: 66c38532da714eceb1ef3488f4f5263d (127.3.89.193:38695)
I20260809 15:35:42.115366 3431 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskm314YR/build/debug/bin/kudu as pid 5393
I20260809 15:35:42.115510 3431 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786289732823074-3431-0/minicluster-data/ts-0/wal/instance
I20260809 15:35:42.115811 3431 external_mini_cluster.cc:1428] Setting key e7f148dca83187460a0a37b19af66f2e
I20260809 15:35:42.116374 5318 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.3.89.193@KRBTEST.COM'} at 127.3.89.193:34285
I20260809 15:35:42.124080 5542 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:35:42.115172 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.89.254:33125 (local address 127.3.89.193:36013)
0809 15:35:42.117022 (+ 1850us) negotiation.cc:107] Waiting for socket to connect
0809 15:35:42.117039 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0809 15:35:42.117175 (+ 136us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 15:35:42.117423 (+ 248us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 15:35:42.117426 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 15:35:42.117613 (+ 187us) client_negotiation.cc:189] Negotiated authn=SASL
0809 15:35:42.117703 (+ 90us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:35:42.117709 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:35:42.118448 (+ 739us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:35:42.118450 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:35:42.119070 (+ 620us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:35:42.119084 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:35:42.119605 (+ 521us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:35:42.119610 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:35:42.119735 (+ 125us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:35:42.120204 (+ 469us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 15:35:42.120221 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0809 15:35:42.120647 (+ 426us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 15:35:42.122629 (+ 1982us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 15:35:42.122632 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 15:35:42.122633 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0809 15:35:42.122813 (+ 180us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 15:35:42.123141 (+ 328us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 15:35:42.123143 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 15:35:42.123144 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0809 15:35:42.123220 (+ 76us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 15:35:42.123678 (+ 458us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 15:35:42.123680 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 15:35:42.123751 (+ 71us) client_negotiation.cc:769] Sending connection context
0809 15:35:42.123827 (+ 76us) client_negotiation.cc:240] Negotiation successful
0809 15:35:42.123931 (+ 104us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1759,"thread_start_us":289,"threads_started":1}
I20260809 15:35:42.124282 5370 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:35:42.115271 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.89.193:36013 (local address 127.3.89.254:33125)
0809 15:35:42.115434 (+ 163us) server_negotiation.cc:206] Beginning negotiation
0809 15:35:42.115440 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 15:35:42.117109 (+ 1669us) server_negotiation.cc:407] Connection header received
0809 15:35:42.117277 (+ 168us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 15:35:42.117280 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 15:35:42.117326 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 15:35:42.117407 (+ 81us) server_negotiation.cc:226] Negotiated authn=SASL
0809 15:35:42.117818 (+ 411us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:35:42.118311 (+ 493us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:35:42.119218 (+ 907us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:35:42.119486 (+ 268us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:35:42.119571 (+ 85us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:35:42.120794 (+ 1223us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 15:35:42.120801 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 15:35:42.120807 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 15:35:42.120838 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0809 15:35:42.122516 (+ 1678us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 15:35:42.122917 (+ 401us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 15:35:42.122924 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 15:35:42.122928 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 15:35:42.123043 (+ 115us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 15:35:42.123317 (+ 274us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 15:35:42.123321 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 15:35:42.123325 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 15:35:42.123570 (+ 245us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 15:35:42.123683 (+ 113us) server_negotiation.cc:1035] Waiting for connection context
0809 15:35:42.123923 (+ 240us) server_negotiation.cc:299] Negotiation successful
0809 15:35:42.124053 (+ 130us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":91}
WARNING: no policy specified for kudu/127.3.89.194@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.3.89.194@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.3.89.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786289732823074-3431-0/krb5kdc/kudu_127.3.89.194.keytab.
Entry for principal kudu/127.3.89.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786289732823074-3431-0/krb5kdc/kudu_127.3.89.194.keytab.
I20260809 15:35:42.145193 3431 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskm314YR/build/debug/bin/kudu
/tmp/dist-test-taskm314YR/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786289732823074-3431-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786289732823074-3431-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786289732823074-3431-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786289732823074-3431-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.3.89.194:0
--local_ip_for_outbound_sockets=127.3.89.194
--webserver_interface=127.3.89.194
--webserver_port=0
--tserver_master_addrs=127.3.89.254:33125
--keytab_file=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786289732823074-3431-0/krb5kdc/kudu_127.3.89.194.keytab
--principal=kudu/127.3.89.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.3.89.212:42515
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true
--txn_keepalive_interval_ms=500 with env {KRB5CCNAME=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786289732823074-3431-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786289732823074-3431-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786289732823074-3431-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 15:35:42.254869 5546 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 15:35:42.255162 5546 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 15:35:42.255210 5546 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 15:35:42.259102 5546 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 15:35:42.259151 5546 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 15:35:42.259189 5546 flags.cc:425] Enabled experimental flag: --txn_keepalive_interval_ms=500
W20260809 15:35:42.259203 5546 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260809 15:35:42.259233 5546 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.3.89.194
I20260809 15:35:42.264310 5546 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.3.89.212:42515
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786289732823074-3431-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786289732823074-3431-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786289732823074-3431-0/krb5kdc/kudu_127.3.89.194.keytab
--principal=kudu/127.3.89.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.3.89.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786289732823074-3431-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.3.89.194
--webserver_port=0
--txn_keepalive_interval_ms=500
--enable_txn_system_client_init=true
--tserver_master_addrs=127.3.89.254:33125
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.5546
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.3.89.194
--log_dir=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786289732823074-3431-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 09 Aug 2026 14:46:55 UTC on ccycloud.quasar-yswkmf.root.comops.site
build id 4576540
I20260809 15:35:42.265189 5546 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 15:35:42.266109 5546 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 15:35:42.275381 5553 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 15:35:42.275403 5555 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 15:35:42.275607 5552 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 15:35:42.275738 5546 server_base.cc:1034] running on GCE node
I20260809 15:35:42.276098 5546 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 15:35:42.276717 5546 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 15:35:42.277874 5546 hybrid_clock.cc:648] HybridClock initialized: now 1786289742277856 us; error 30 us; skew 500 ppm
Aug 09 15:35:42 dist-test-slave-2w3w krb5kdc[5270](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786289742, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.89.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 15:35:42.285075 5546 init.cc:377] Logged in from keytab as kudu/127.3.89.194@KRBTEST.COM (short username kudu)
I20260809 15:35:42.285975 5546 webserver.cc:460] Webserver started at http://127.3.89.194:34993/ using document root <none> and password file <none>
I20260809 15:35:42.286514 5546 fs_manager.cc:359] Metadata directory not provided
I20260809 15:35:42.286602 5546 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 15:35:42.286849 5546 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 15:35:42.288820 5546 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786289732823074-3431-0/minicluster-data/ts-1/data/instance:
uuid: "31f3c6c99bab401fb6eb011dea8fb13b"
format_stamp: "Formatted at 2026-08-09 15:35:42 on dist-test-slave-2w3w"
server_key: "924fa204fd891d3728d52b6d135fea21"
server_key_iv: "414c03aea617b155e75cea270bcfc303"
server_key_version: "encryptionkey@0"
I20260809 15:35:42.289412 5546 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786289732823074-3431-0/minicluster-data/ts-1/wal/instance:
uuid: "31f3c6c99bab401fb6eb011dea8fb13b"
format_stamp: "Formatted at 2026-08-09 15:35:42 on dist-test-slave-2w3w"
server_key: "924fa204fd891d3728d52b6d135fea21"
server_key_iv: "414c03aea617b155e75cea270bcfc303"
server_key_version: "encryptionkey@0"
I20260809 15:35:42.292996 5546 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.005s sys 0.000s
I20260809 15:35:42.295372 5563 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 15:35:42.296517 5546 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260809 15:35:42.296653 5546 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786289732823074-3431-0/minicluster-data/ts-1/data,/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786289732823074-3431-0/minicluster-data/ts-1/wal
uuid: "31f3c6c99bab401fb6eb011dea8fb13b"
format_stamp: "Formatted at 2026-08-09 15:35:42 on dist-test-slave-2w3w"
server_key: "924fa204fd891d3728d52b6d135fea21"
server_key_iv: "414c03aea617b155e75cea270bcfc303"
server_key_version: "encryptionkey@0"
I20260809 15:35:42.296757 5546 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786289732823074-3431-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786289732823074-3431-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786289732823074-3431-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 15:35:42.315073 5546 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 15:35:42.317822 5546 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 15:35:42.318028 5546 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 15:35:42.319530 5546 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 15:35:42.319603 5546 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 15:35:42.319679 5546 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 15:35:42.319722 5546 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 09 15:35:42 dist-test-slave-2w3w krb5kdc[5270](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786289742, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.89.194@KRBTEST.COM for kudu/127.3.89.254@KRBTEST.COM
I20260809 15:35:42.330322 5546 rpc_server.cc:307] RPC server started. Bound to: 127.3.89.194:37671
I20260809 15:35:42.330511 5688 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.3.89.194:37671 every 8 connection(s)
I20260809 15:35:42.331291 5546 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786289732823074-3431-0/minicluster-data/ts-1/data/info.pb
I20260809 15:35:42.333449 5370 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:35:42.320204 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.89.194:50373 (local address 127.3.89.254:33125)
0809 15:35:42.320381 (+ 177us) server_negotiation.cc:206] Beginning negotiation
0809 15:35:42.320386 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 15:35:42.321768 (+ 1382us) server_negotiation.cc:407] Connection header received
0809 15:35:42.323304 (+ 1536us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 15:35:42.323309 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 15:35:42.323368 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 15:35:42.323490 (+ 122us) server_negotiation.cc:226] Negotiated authn=SASL
0809 15:35:42.325149 (+ 1659us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:35:42.325780 (+ 631us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:35:42.326927 (+ 1147us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:35:42.327180 (+ 253us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:35:42.327301 (+ 121us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:35:42.329701 (+ 2400us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 15:35:42.329705 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 15:35:42.329707 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 15:35:42.329734 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0809 15:35:42.331500 (+ 1766us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 15:35:42.332084 (+ 584us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 15:35:42.332088 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 15:35:42.332089 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 15:35:42.332184 (+ 95us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 15:35:42.332562 (+ 378us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 15:35:42.332564 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 15:35:42.332565 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 15:35:42.332736 (+ 171us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 15:35:42.332814 (+ 78us) server_negotiation.cc:1035] Waiting for connection context
0809 15:35:42.333236 (+ 422us) server_negotiation.cc:299] Negotiation successful
0809 15:35:42.333332 (+ 96us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":62}
I20260809 15:35:42.334215 5574 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:35:42.321081 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.89.254:33125 (local address 127.3.89.194:50373)
0809 15:35:42.321606 (+ 525us) negotiation.cc:107] Waiting for socket to connect
0809 15:35:42.321661 (+ 55us) client_negotiation.cc:174] Beginning negotiation
0809 15:35:42.323036 (+ 1375us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 15:35:42.323712 (+ 676us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 15:35:42.323726 (+ 14us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 15:35:42.324252 (+ 526us) client_negotiation.cc:189] Negotiated authn=SASL
0809 15:35:42.324981 (+ 729us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:35:42.325002 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:35:42.325918 (+ 916us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:35:42.325922 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:35:42.326790 (+ 868us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:35:42.326800 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:35:42.327332 (+ 532us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:35:42.327336 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:35:42.327513 (+ 177us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:35:42.328077 (+ 564us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 15:35:42.328093 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0809 15:35:42.329518 (+ 1425us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 15:35:42.331614 (+ 2096us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 15:35:42.331620 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 15:35:42.331632 (+ 12us) client_negotiation.cc:760] calling sasl_client_step()
0809 15:35:42.331987 (+ 355us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 15:35:42.332306 (+ 319us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 15:35:42.332309 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 15:35:42.332311 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 15:35:42.332477 (+ 166us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 15:35:42.332832 (+ 355us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 15:35:42.332836 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 15:35:42.333036 (+ 200us) client_negotiation.cc:769] Sending connection context
0809 15:35:42.333185 (+ 149us) client_negotiation.cc:240] Negotiation successful
0809 15:35:42.333421 (+ 236us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":297,"thread_start_us":137,"threads_started":1}
I20260809 15:35:42.335417 5689 heartbeater.cc:347] Connected to a master server at 127.3.89.254:33125
I20260809 15:35:42.335573 5689 heartbeater.cc:464] Registering TS with master...
I20260809 15:35:42.336028 5689 heartbeater.cc:511] Master 127.3.89.254:33125 requested a full tablet report, sending...
I20260809 15:35:42.337091 5318 ts_manager.cc:194] Registered new tserver with Master: 31f3c6c99bab401fb6eb011dea8fb13b (127.3.89.194:37671)
I20260809 15:35:42.337882 5318 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.3.89.194@KRBTEST.COM'} at 127.3.89.194:50373
I20260809 15:35:42.341233 3431 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskm314YR/build/debug/bin/kudu as pid 5546
I20260809 15:35:42.341334 3431 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786289732823074-3431-0/minicluster-data/ts-1/wal/instance
I20260809 15:35:42.341687 3431 external_mini_cluster.cc:1428] Setting key b865882ed7a3371d02ff01473975c00b
I20260809 15:35:42.344504 5370 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:35:42.337507 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.89.194:55587 (local address 127.3.89.254:33125)
0809 15:35:42.337632 (+ 125us) server_negotiation.cc:206] Beginning negotiation
0809 15:35:42.337636 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 15:35:42.337771 (+ 135us) server_negotiation.cc:407] Connection header received
0809 15:35:42.337857 (+ 86us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 15:35:42.337860 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 15:35:42.337906 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 15:35:42.337987 (+ 81us) server_negotiation.cc:226] Negotiated authn=SASL
0809 15:35:42.338468 (+ 481us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:35:42.339027 (+ 559us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:35:42.339837 (+ 810us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:35:42.340058 (+ 221us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:35:42.340164 (+ 106us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:35:42.341117 (+ 953us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 15:35:42.341120 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 15:35:42.341121 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 15:35:42.341142 (+ 21us) server_negotiation.cc:922] calling sasl_server_start()
0809 15:35:42.342837 (+ 1695us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 15:35:42.343356 (+ 519us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 15:35:42.343359 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 15:35:42.343360 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 15:35:42.343432 (+ 72us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 15:35:42.343726 (+ 294us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 15:35:42.343728 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 15:35:42.343729 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 15:35:42.343935 (+ 206us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 15:35:42.344015 (+ 80us) server_negotiation.cc:1035] Waiting for connection context
0809 15:35:42.344319 (+ 304us) server_negotiation.cc:299] Negotiation successful
0809 15:35:42.344403 (+ 84us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":50}
I20260809 15:35:42.344801 5695 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:35:42.337440 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.89.254:33125 (local address 127.3.89.194:55587)
0809 15:35:42.337688 (+ 248us) negotiation.cc:107] Waiting for socket to connect
0809 15:35:42.337707 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0809 15:35:42.337782 (+ 75us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 15:35:42.338024 (+ 242us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 15:35:42.338029 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 15:35:42.338255 (+ 226us) client_negotiation.cc:189] Negotiated authn=SASL
0809 15:35:42.338358 (+ 103us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:35:42.338363 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:35:42.339152 (+ 789us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:35:42.339155 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:35:42.339717 (+ 562us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:35:42.339725 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:35:42.340167 (+ 442us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:35:42.340171 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:35:42.340259 (+ 88us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:35:42.340660 (+ 401us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 15:35:42.340670 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0809 15:35:42.340948 (+ 278us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 15:35:42.342987 (+ 2039us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 15:35:42.342994 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 15:35:42.342999 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0809 15:35:42.343253 (+ 254us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 15:35:42.343528 (+ 275us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 15:35:42.343531 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 15:35:42.343533 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 15:35:42.343635 (+ 102us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 15:35:42.344036 (+ 401us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 15:35:42.344042 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 15:35:42.344152 (+ 110us) client_negotiation.cc:769] Sending connection context
0809 15:35:42.344273 (+ 121us) client_negotiation.cc:240] Negotiation successful
0809 15:35:42.344439 (+ 166us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":173,"thread_start_us":97,"threads_started":1}
WARNING: no policy specified for kudu/127.3.89.195@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.3.89.195@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.3.89.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786289732823074-3431-0/krb5kdc/kudu_127.3.89.195.keytab.
Entry for principal kudu/127.3.89.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786289732823074-3431-0/krb5kdc/kudu_127.3.89.195.keytab.
I20260809 15:35:42.369700 3431 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskm314YR/build/debug/bin/kudu
/tmp/dist-test-taskm314YR/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786289732823074-3431-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786289732823074-3431-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786289732823074-3431-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786289732823074-3431-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.3.89.195:0
--local_ip_for_outbound_sockets=127.3.89.195
--webserver_interface=127.3.89.195
--webserver_port=0
--tserver_master_addrs=127.3.89.254:33125
--keytab_file=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786289732823074-3431-0/krb5kdc/kudu_127.3.89.195.keytab
--principal=kudu/127.3.89.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.3.89.212:42515
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true
--txn_keepalive_interval_ms=500 with env {KRB5CCNAME=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786289732823074-3431-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786289732823074-3431-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786289732823074-3431-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 15:35:42.497049 5698 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 15:35:42.497299 5698 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 15:35:42.497345 5698 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 15:35:42.501430 5698 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 15:35:42.501485 5698 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 15:35:42.501524 5698 flags.cc:425] Enabled experimental flag: --txn_keepalive_interval_ms=500
W20260809 15:35:42.501539 5698 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260809 15:35:42.501575 5698 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.3.89.195
I20260809 15:35:42.506927 5698 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.3.89.212:42515
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786289732823074-3431-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786289732823074-3431-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786289732823074-3431-0/krb5kdc/kudu_127.3.89.195.keytab
--principal=kudu/127.3.89.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.3.89.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786289732823074-3431-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.3.89.195
--webserver_port=0
--txn_keepalive_interval_ms=500
--enable_txn_system_client_init=true
--tserver_master_addrs=127.3.89.254:33125
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.5698
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.3.89.195
--log_dir=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786289732823074-3431-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 09 Aug 2026 14:46:55 UTC on ccycloud.quasar-yswkmf.root.comops.site
build id 4576540
I20260809 15:35:42.508092 5698 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 15:35:42.509033 5698 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 15:35:42.518608 5705 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 15:35:42.518708 5698 server_base.cc:1034] running on GCE node
W20260809 15:35:42.518586 5707 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 15:35:42.518589 5704 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 15:35:42.519330 5698 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 15:35:42.519919 5698 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 15:35:42.521099 5698 hybrid_clock.cc:648] HybridClock initialized: now 1786289742521070 us; error 42 us; skew 500 ppm
Aug 09 15:35:42 dist-test-slave-2w3w krb5kdc[5270](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786289742, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.89.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 15:35:42.523759 5698 init.cc:377] Logged in from keytab as kudu/127.3.89.195@KRBTEST.COM (short username kudu)
I20260809 15:35:42.524526 5698 webserver.cc:460] Webserver started at http://127.3.89.195:33495/ using document root <none> and password file <none>
I20260809 15:35:42.525017 5698 fs_manager.cc:359] Metadata directory not provided
I20260809 15:35:42.525115 5698 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 15:35:42.525315 5698 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 15:35:42.527670 5698 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786289732823074-3431-0/minicluster-data/ts-2/data/instance:
uuid: "8ed96760f4584a5ca96e33c6ce9a8f2e"
format_stamp: "Formatted at 2026-08-09 15:35:42 on dist-test-slave-2w3w"
server_key: "b061c3fb9904a99d471b5ad9ff88f445"
server_key_iv: "e8ad0dd21408166d1de6323f766aad78"
server_key_version: "encryptionkey@0"
I20260809 15:35:42.528173 5698 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786289732823074-3431-0/minicluster-data/ts-2/wal/instance:
uuid: "8ed96760f4584a5ca96e33c6ce9a8f2e"
format_stamp: "Formatted at 2026-08-09 15:35:42 on dist-test-slave-2w3w"
server_key: "b061c3fb9904a99d471b5ad9ff88f445"
server_key_iv: "e8ad0dd21408166d1de6323f766aad78"
server_key_version: "encryptionkey@0"
I20260809 15:35:42.531605 5698 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.003s sys 0.001s
I20260809 15:35:42.533804 5715 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 15:35:42.534996 5698 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260809 15:35:42.535097 5698 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786289732823074-3431-0/minicluster-data/ts-2/data,/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786289732823074-3431-0/minicluster-data/ts-2/wal
uuid: "8ed96760f4584a5ca96e33c6ce9a8f2e"
format_stamp: "Formatted at 2026-08-09 15:35:42 on dist-test-slave-2w3w"
server_key: "b061c3fb9904a99d471b5ad9ff88f445"
server_key_iv: "e8ad0dd21408166d1de6323f766aad78"
server_key_version: "encryptionkey@0"
I20260809 15:35:42.535177 5698 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786289732823074-3431-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786289732823074-3431-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786289732823074-3431-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 15:35:42.553740 5698 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 15:35:42.556565 5698 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 15:35:42.556751 5698 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 15:35:42.558084 5698 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 15:35:42.558156 5698 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 15:35:42.558220 5698 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 15:35:42.558264 5698 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 09 15:35:42 dist-test-slave-2w3w krb5kdc[5270](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786289742, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.89.195@KRBTEST.COM for kudu/127.3.89.254@KRBTEST.COM
I20260809 15:35:42.569321 5698 rpc_server.cc:307] RPC server started. Bound to: 127.3.89.195:41519
I20260809 15:35:42.570122 5698 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786289732823074-3431-0/minicluster-data/ts-2/data/info.pb
I20260809 15:35:42.570564 5841 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.3.89.195:41519 every 8 connection(s)
I20260809 15:35:42.572109 5370 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:35:42.559006 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.89.195:36459 (local address 127.3.89.254:33125)
0809 15:35:42.559136 (+ 130us) server_negotiation.cc:206] Beginning negotiation
0809 15:35:42.559140 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 15:35:42.560082 (+ 942us) server_negotiation.cc:407] Connection header received
0809 15:35:42.561151 (+ 1069us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 15:35:42.561155 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 15:35:42.561220 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 15:35:42.561309 (+ 89us) server_negotiation.cc:226] Negotiated authn=SASL
0809 15:35:42.563133 (+ 1824us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:35:42.563781 (+ 648us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:35:42.565040 (+ 1259us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:35:42.565320 (+ 280us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:35:42.565436 (+ 116us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:35:42.568196 (+ 2760us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 15:35:42.568200 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 15:35:42.568203 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 15:35:42.568235 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0809 15:35:42.569858 (+ 1623us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 15:35:42.570465 (+ 607us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 15:35:42.570470 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 15:35:42.570472 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 15:35:42.570568 (+ 96us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 15:35:42.570994 (+ 426us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 15:35:42.570998 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 15:35:42.570999 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 15:35:42.571225 (+ 226us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 15:35:42.571327 (+ 102us) server_negotiation.cc:1035] Waiting for connection context
0809 15:35:42.571876 (+ 549us) server_negotiation.cc:299] Negotiation successful
0809 15:35:42.571995 (+ 119us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":50}
I20260809 15:35:42.572847 5725 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:35:42.559465 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.89.254:33125 (local address 127.3.89.195:36459)
0809 15:35:42.559937 (+ 472us) negotiation.cc:107] Waiting for socket to connect
0809 15:35:42.559983 (+ 46us) client_negotiation.cc:174] Beginning negotiation
0809 15:35:42.560895 (+ 912us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 15:35:42.561757 (+ 862us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 15:35:42.561769 (+ 12us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 15:35:42.562177 (+ 408us) client_negotiation.cc:189] Negotiated authn=SASL
0809 15:35:42.562906 (+ 729us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:35:42.562930 (+ 24us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:35:42.563915 (+ 985us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:35:42.563918 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:35:42.564909 (+ 991us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:35:42.564918 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:35:42.565482 (+ 564us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:35:42.565486 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:35:42.565653 (+ 167us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:35:42.566215 (+ 562us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 15:35:42.566235 (+ 20us) client_negotiation.cc:656] calling sasl_client_start()
0809 15:35:42.568019 (+ 1784us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 15:35:42.569983 (+ 1964us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 15:35:42.569992 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 15:35:42.570008 (+ 16us) client_negotiation.cc:760] calling sasl_client_step()
0809 15:35:42.570347 (+ 339us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 15:35:42.570681 (+ 334us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 15:35:42.570686 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 15:35:42.570691 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0809 15:35:42.570844 (+ 153us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 15:35:42.571408 (+ 564us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 15:35:42.571412 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 15:35:42.571609 (+ 197us) client_negotiation.cc:769] Sending connection context
0809 15:35:42.571750 (+ 141us) client_negotiation.cc:240] Negotiation successful
0809 15:35:42.572041 (+ 291us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":299,"thread_start_us":132,"threads_started":1}
I20260809 15:35:42.574050 5842 heartbeater.cc:347] Connected to a master server at 127.3.89.254:33125
I20260809 15:35:42.574191 5842 heartbeater.cc:464] Registering TS with master...
I20260809 15:35:42.574740 5842 heartbeater.cc:511] Master 127.3.89.254:33125 requested a full tablet report, sending...
I20260809 15:35:42.575762 5319 ts_manager.cc:194] Registered new tserver with Master: 8ed96760f4584a5ca96e33c6ce9a8f2e (127.3.89.195:41519)
I20260809 15:35:42.576426 3431 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskm314YR/build/debug/bin/kudu as pid 5698
I20260809 15:35:42.576574 3431 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786289732823074-3431-0/minicluster-data/ts-2/wal/instance
I20260809 15:35:42.576575 5319 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.3.89.195@KRBTEST.COM'} at 127.3.89.195:36459
I20260809 15:35:42.576938 3431 external_mini_cluster.cc:1428] Setting key 9a4be9d1b32e83b76d3170f3d5a2de6f
I20260809 15:35:42.579097 3431 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260809 15:35:42.584106 5850 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:35:42.575994 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.89.254:33125 (local address 127.3.89.195:54515)
0809 15:35:42.576319 (+ 325us) negotiation.cc:107] Waiting for socket to connect
0809 15:35:42.576333 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0809 15:35:42.576466 (+ 133us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 15:35:42.576812 (+ 346us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 15:35:42.576814 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 15:35:42.577031 (+ 217us) client_negotiation.cc:189] Negotiated authn=SASL
0809 15:35:42.577149 (+ 118us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:35:42.577156 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:35:42.578270 (+ 1114us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:35:42.578272 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:35:42.579119 (+ 847us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:35:42.579130 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:35:42.579630 (+ 500us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:35:42.579634 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:35:42.579722 (+ 88us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:35:42.580251 (+ 529us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 15:35:42.580267 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0809 15:35:42.580558 (+ 291us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 15:35:42.582633 (+ 2075us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 15:35:42.582635 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 15:35:42.582637 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 15:35:42.582822 (+ 185us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 15:35:42.583185 (+ 363us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 15:35:42.583188 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 15:35:42.583189 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0809 15:35:42.583285 (+ 96us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 15:35:42.583725 (+ 440us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 15:35:42.583728 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 15:35:42.583798 (+ 70us) client_negotiation.cc:769] Sending connection context
0809 15:35:42.583879 (+ 81us) client_negotiation.cc:240] Negotiation successful
0809 15:35:42.583983 (+ 104us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":253,"thread_start_us":85,"threads_started":1}
I20260809 15:35:42.584482 5370 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:35:42.576076 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.89.195:54515 (local address 127.3.89.254:33125)
0809 15:35:42.576189 (+ 113us) server_negotiation.cc:206] Beginning negotiation
0809 15:35:42.576195 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 15:35:42.576405 (+ 210us) server_negotiation.cc:407] Connection header received
0809 15:35:42.576561 (+ 156us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 15:35:42.576564 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 15:35:42.576610 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 15:35:42.576660 (+ 50us) server_negotiation.cc:226] Negotiated authn=SASL
0809 15:35:42.577451 (+ 791us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:35:42.578099 (+ 648us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:35:42.579260 (+ 1161us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:35:42.579518 (+ 258us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:35:42.579626 (+ 108us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:35:42.580692 (+ 1066us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 15:35:42.580695 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 15:35:42.580698 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 15:35:42.580720 (+ 22us) server_negotiation.cc:922] calling sasl_server_start()
0809 15:35:42.582508 (+ 1788us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 15:35:42.582917 (+ 409us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 15:35:42.582924 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 15:35:42.582925 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 15:35:42.583075 (+ 150us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 15:35:42.583382 (+ 307us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 15:35:42.583388 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 15:35:42.583392 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 15:35:42.583628 (+ 236us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 15:35:42.583736 (+ 108us) server_negotiation.cc:1035] Waiting for connection context
0809 15:35:42.583959 (+ 223us) server_negotiation.cc:299] Negotiation successful
0809 15:35:42.584089 (+ 130us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":42}
Aug 09 15:35:42 dist-test-slave-2w3w krb5kdc[5270](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786289742, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Aug 09 15:35:42 dist-test-slave-2w3w krb5kdc[5270](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786289742, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.3.89.254@KRBTEST.COM
I20260809 15:35:42.607045 5370 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:35:42.598922 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44998 (local address 127.3.89.254:33125)
0809 15:35:42.599098 (+ 176us) server_negotiation.cc:206] Beginning negotiation
0809 15:35:42.599102 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 15:35:42.599115 (+ 13us) server_negotiation.cc:407] Connection header received
0809 15:35:42.599155 (+ 40us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 15:35:42.599157 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 15:35:42.599197 (+ 40us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 15:35:42.599277 (+ 80us) server_negotiation.cc:226] Negotiated authn=SASL
0809 15:35:42.600172 (+ 895us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:35:42.600744 (+ 572us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:35:42.601662 (+ 918us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:35:42.601865 (+ 203us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:35:42.601942 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:35:42.604394 (+ 2452us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 15:35:42.604397 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 15:35:42.604400 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 15:35:42.604422 (+ 22us) server_negotiation.cc:922] calling sasl_server_start()
0809 15:35:42.605615 (+ 1193us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 15:35:42.605985 (+ 370us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 15:35:42.605987 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 15:35:42.605988 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 15:35:42.606061 (+ 73us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 15:35:42.606283 (+ 222us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 15:35:42.606286 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 15:35:42.606287 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 15:35:42.606453 (+ 166us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 15:35:42.606562 (+ 109us) server_negotiation.cc:1035] Waiting for connection context
0809 15:35:42.606769 (+ 207us) server_negotiation.cc:299] Negotiation successful
0809 15:35:42.606861 (+ 92us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":86}
I20260809 15:35:42.609964 5319 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:44998:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260809 15:35:42.613656 5319 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260809 15:35:42.627641 5861 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:35:42.621537 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.89.195:41519 (local address 127.0.0.1:46744)
0809 15:35:42.621898 (+ 361us) negotiation.cc:107] Waiting for socket to connect
0809 15:35:42.621947 (+ 49us) client_negotiation.cc:174] Beginning negotiation
0809 15:35:42.622091 (+ 144us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 15:35:42.623144 (+ 1053us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 15:35:42.623152 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 15:35:42.623172 (+ 20us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 15:35:42.623271 (+ 99us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:35:42.623275 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:35:42.624716 (+ 1441us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:35:42.624720 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:35:42.626101 (+ 1381us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:35:42.626111 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:35:42.627093 (+ 982us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:35:42.627097 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:35:42.627210 (+ 113us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:35:42.627251 (+ 41us) client_negotiation.cc:769] Sending connection context
0809 15:35:42.627370 (+ 119us) client_negotiation.cc:240] Negotiation successful
0809 15:35:42.627488 (+ 118us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":173,"mutex_wait_us":60,"thread_start_us":88,"threads_started":1}
I20260809 15:35:42.627854 5864 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:35:42.621924 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.89.193:38695 (local address 127.0.0.1:54908)
0809 15:35:42.622838 (+ 914us) negotiation.cc:107] Waiting for socket to connect
0809 15:35:42.622856 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0809 15:35:42.622950 (+ 94us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 15:35:42.623705 (+ 755us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 15:35:42.623708 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 15:35:42.623726 (+ 18us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 15:35:42.623835 (+ 109us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:35:42.623843 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:35:42.625076 (+ 1233us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:35:42.625079 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:35:42.626351 (+ 1272us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:35:42.626362 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:35:42.627419 (+ 1057us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:35:42.627422 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:35:42.627524 (+ 102us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:35:42.627546 (+ 22us) client_negotiation.cc:769] Sending connection context
0809 15:35:42.627655 (+ 109us) client_negotiation.cc:240] Negotiation successful
0809 15:35:42.627727 (+ 72us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":206,"mutex_wait_us":1,"thread_start_us":107,"threads_started":1}
I20260809 15:35:42.627894 5866 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:35:42.622342 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54908 (local address 127.3.89.193:38695)
0809 15:35:42.623374 (+ 1032us) server_negotiation.cc:206] Beginning negotiation
0809 15:35:42.623379 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 15:35:42.623395 (+ 16us) server_negotiation.cc:407] Connection header received
0809 15:35:42.623452 (+ 57us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 15:35:42.623455 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 15:35:42.623577 (+ 122us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 15:35:42.623723 (+ 146us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 15:35:42.623971 (+ 248us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:35:42.624942 (+ 971us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:35:42.626495 (+ 1553us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:35:42.627281 (+ 786us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:35:42.627406 (+ 125us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:35:42.627548 (+ 142us) server_negotiation.cc:1035] Waiting for connection context
0809 15:35:42.627687 (+ 139us) server_negotiation.cc:299] Negotiation successful
0809 15:35:42.627776 (+ 89us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":947,"thread_start_us":68,"threads_started":1}
I20260809 15:35:42.628342 5865 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:35:42.622245 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.89.194:37671 (local address 127.0.0.1:42122)
0809 15:35:42.623071 (+ 826us) negotiation.cc:107] Waiting for socket to connect
0809 15:35:42.623085 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0809 15:35:42.623171 (+ 86us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 15:35:42.623994 (+ 823us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 15:35:42.623996 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 15:35:42.624017 (+ 21us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 15:35:42.624095 (+ 78us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:35:42.624100 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:35:42.625321 (+ 1221us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:35:42.625324 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:35:42.626505 (+ 1181us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:35:42.626511 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:35:42.627848 (+ 1337us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:35:42.627855 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:35:42.628063 (+ 208us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:35:42.628085 (+ 22us) client_negotiation.cc:769] Sending connection context
0809 15:35:42.628145 (+ 60us) client_negotiation.cc:240] Negotiation successful
0809 15:35:42.628211 (+ 66us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":761,"thread_start_us":84,"threads_started":1}
I20260809 15:35:42.628448 5863 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:35:42.621661 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46744 (local address 127.3.89.195:41519)
0809 15:35:42.622538 (+ 877us) server_negotiation.cc:206] Beginning negotiation
0809 15:35:42.622551 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0809 15:35:42.622576 (+ 25us) server_negotiation.cc:407] Connection header received
0809 15:35:42.622654 (+ 78us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 15:35:42.622659 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 15:35:42.622868 (+ 209us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 15:35:42.623087 (+ 219us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 15:35:42.623403 (+ 316us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:35:42.624594 (+ 1191us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:35:42.626232 (+ 1638us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:35:42.626953 (+ 721us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:35:42.627094 (+ 141us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:35:42.627256 (+ 162us) server_negotiation.cc:1035] Waiting for connection context
0809 15:35:42.628176 (+ 920us) server_negotiation.cc:299] Negotiation successful
0809 15:35:42.628268 (+ 92us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":764,"thread_start_us":107,"threads_started":1}
I20260809 15:35:42.630486 5773 tablet_service.cc:1467] Processing CreateTablet for tablet 0285927261164c3581d7308de0043ffb (DEFAULT_TABLE table=test-table [id=04331610cf50454694403002c6b87934]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 15:35:42.630455 5465 tablet_service.cc:1467] Processing CreateTablet for tablet 0285927261164c3581d7308de0043ffb (DEFAULT_TABLE table=test-table [id=04331610cf50454694403002c6b87934]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 15:35:42.631314 5867 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:35:42.622650 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42122 (local address 127.3.89.194:37671)
0809 15:35:42.623584 (+ 934us) server_negotiation.cc:206] Beginning negotiation
0809 15:35:42.623596 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0809 15:35:42.623617 (+ 21us) server_negotiation.cc:407] Connection header received
0809 15:35:42.623686 (+ 69us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 15:35:42.623691 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 15:35:42.623863 (+ 172us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 15:35:42.623997 (+ 134us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 15:35:42.624222 (+ 225us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:35:42.625180 (+ 958us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:35:42.626612 (+ 1432us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:35:42.627457 (+ 845us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:35:42.627550 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:35:42.627687 (+ 137us) server_negotiation.cc:1035] Waiting for connection context
0809 15:35:42.631067 (+ 3380us) server_negotiation.cc:299] Negotiation successful
0809 15:35:42.631165 (+ 98us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":832,"thread_start_us":131,"threads_started":1}
I20260809 15:35:42.631464 5773 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 0285927261164c3581d7308de0043ffb. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 15:35:42.631475 5465 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 0285927261164c3581d7308de0043ffb. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 15:35:42.637040 5615 tablet_service.cc:1467] Processing CreateTablet for tablet 0285927261164c3581d7308de0043ffb (DEFAULT_TABLE table=test-table [id=04331610cf50454694403002c6b87934]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 15:35:42.637652 5869 tablet_bootstrap.cc:492] T 0285927261164c3581d7308de0043ffb P 66c38532da714eceb1ef3488f4f5263d: Bootstrap starting.
I20260809 15:35:42.638015 5615 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 0285927261164c3581d7308de0043ffb. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 15:35:42.639822 5869 tablet_bootstrap.cc:654] T 0285927261164c3581d7308de0043ffb P 66c38532da714eceb1ef3488f4f5263d: Neither blocks nor log segments found. Creating new log.
I20260809 15:35:42.640565 5869 log.cc:824] T 0285927261164c3581d7308de0043ffb P 66c38532da714eceb1ef3488f4f5263d: Log is configured to *not* fsync() on all Append() calls
I20260809 15:35:42.640895 5870 tablet_bootstrap.cc:492] T 0285927261164c3581d7308de0043ffb P 8ed96760f4584a5ca96e33c6ce9a8f2e: Bootstrap starting.
I20260809 15:35:42.642506 5870 tablet_bootstrap.cc:654] T 0285927261164c3581d7308de0043ffb P 8ed96760f4584a5ca96e33c6ce9a8f2e: Neither blocks nor log segments found. Creating new log.
I20260809 15:35:42.643402 5870 log.cc:824] T 0285927261164c3581d7308de0043ffb P 8ed96760f4584a5ca96e33c6ce9a8f2e: Log is configured to *not* fsync() on all Append() calls
I20260809 15:35:42.644277 5869 tablet_bootstrap.cc:492] T 0285927261164c3581d7308de0043ffb P 66c38532da714eceb1ef3488f4f5263d: No bootstrap required, opened a new log
I20260809 15:35:42.644479 5869 ts_tablet_manager.cc:1397] T 0285927261164c3581d7308de0043ffb P 66c38532da714eceb1ef3488f4f5263d: Time spent bootstrapping tablet: real 0.007s user 0.004s sys 0.000s
I20260809 15:35:42.645291 5870 tablet_bootstrap.cc:492] T 0285927261164c3581d7308de0043ffb P 8ed96760f4584a5ca96e33c6ce9a8f2e: No bootstrap required, opened a new log
I20260809 15:35:42.645433 5873 tablet_bootstrap.cc:492] T 0285927261164c3581d7308de0043ffb P 31f3c6c99bab401fb6eb011dea8fb13b: Bootstrap starting.
I20260809 15:35:42.645480 5870 ts_tablet_manager.cc:1397] T 0285927261164c3581d7308de0043ffb P 8ed96760f4584a5ca96e33c6ce9a8f2e: Time spent bootstrapping tablet: real 0.005s user 0.003s sys 0.000s
I20260809 15:35:42.647235 5873 tablet_bootstrap.cc:654] T 0285927261164c3581d7308de0043ffb P 31f3c6c99bab401fb6eb011dea8fb13b: Neither blocks nor log segments found. Creating new log.
I20260809 15:35:42.647907 5873 log.cc:824] T 0285927261164c3581d7308de0043ffb P 31f3c6c99bab401fb6eb011dea8fb13b: Log is configured to *not* fsync() on all Append() calls
I20260809 15:35:42.648952 5869 raft_consensus.cc:348] T 0285927261164c3581d7308de0043ffb P 66c38532da714eceb1ef3488f4f5263d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8ed96760f4584a5ca96e33c6ce9a8f2e" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41519 } } peers { permanent_uuid: "31f3c6c99bab401fb6eb011dea8fb13b" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 37671 } } peers { permanent_uuid: "66c38532da714eceb1ef3488f4f5263d" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 38695 } }
I20260809 15:35:42.649237 5869 raft_consensus.cc:374] T 0285927261164c3581d7308de0043ffb P 66c38532da714eceb1ef3488f4f5263d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 15:35:42.649294 5869 raft_consensus.cc:729] T 0285927261164c3581d7308de0043ffb P 66c38532da714eceb1ef3488f4f5263d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 66c38532da714eceb1ef3488f4f5263d, State: Initialized, Role: FOLLOWER
I20260809 15:35:42.649354 5870 raft_consensus.cc:348] T 0285927261164c3581d7308de0043ffb P 8ed96760f4584a5ca96e33c6ce9a8f2e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8ed96760f4584a5ca96e33c6ce9a8f2e" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41519 } } peers { permanent_uuid: "31f3c6c99bab401fb6eb011dea8fb13b" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 37671 } } peers { permanent_uuid: "66c38532da714eceb1ef3488f4f5263d" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 38695 } }
I20260809 15:35:42.649595 5873 tablet_bootstrap.cc:492] T 0285927261164c3581d7308de0043ffb P 31f3c6c99bab401fb6eb011dea8fb13b: No bootstrap required, opened a new log
I20260809 15:35:42.649641 5870 raft_consensus.cc:374] T 0285927261164c3581d7308de0043ffb P 8ed96760f4584a5ca96e33c6ce9a8f2e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 15:35:42.649708 5870 raft_consensus.cc:729] T 0285927261164c3581d7308de0043ffb P 8ed96760f4584a5ca96e33c6ce9a8f2e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8ed96760f4584a5ca96e33c6ce9a8f2e, State: Initialized, Role: FOLLOWER
I20260809 15:35:42.649814 5873 ts_tablet_manager.cc:1397] T 0285927261164c3581d7308de0043ffb P 31f3c6c99bab401fb6eb011dea8fb13b: Time spent bootstrapping tablet: real 0.005s user 0.003s sys 0.000s
I20260809 15:35:42.649788 5869 consensus_queue.cc:260] T 0285927261164c3581d7308de0043ffb P 66c38532da714eceb1ef3488f4f5263d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8ed96760f4584a5ca96e33c6ce9a8f2e" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41519 } } peers { permanent_uuid: "31f3c6c99bab401fb6eb011dea8fb13b" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 37671 } } peers { permanent_uuid: "66c38532da714eceb1ef3488f4f5263d" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 38695 } }
I20260809 15:35:42.650091 5870 consensus_queue.cc:260] T 0285927261164c3581d7308de0043ffb P 8ed96760f4584a5ca96e33c6ce9a8f2e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8ed96760f4584a5ca96e33c6ce9a8f2e" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41519 } } peers { permanent_uuid: "31f3c6c99bab401fb6eb011dea8fb13b" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 37671 } } peers { permanent_uuid: "66c38532da714eceb1ef3488f4f5263d" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 38695 } }
I20260809 15:35:42.650687 5869 ts_tablet_manager.cc:1428] T 0285927261164c3581d7308de0043ffb P 66c38532da714eceb1ef3488f4f5263d: Time spent starting tablet: real 0.006s user 0.006s sys 0.000s
I20260809 15:35:42.650911 5870 ts_tablet_manager.cc:1428] T 0285927261164c3581d7308de0043ffb P 8ed96760f4584a5ca96e33c6ce9a8f2e: Time spent starting tablet: real 0.005s user 0.006s sys 0.000s
I20260809 15:35:42.651069 5536 heartbeater.cc:503] Master 127.3.89.254:33125 was elected leader, sending a full tablet report...
I20260809 15:35:42.651242 5842 heartbeater.cc:503] Master 127.3.89.254:33125 was elected leader, sending a full tablet report...
I20260809 15:35:42.653347 5873 raft_consensus.cc:348] T 0285927261164c3581d7308de0043ffb P 31f3c6c99bab401fb6eb011dea8fb13b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8ed96760f4584a5ca96e33c6ce9a8f2e" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41519 } } peers { permanent_uuid: "31f3c6c99bab401fb6eb011dea8fb13b" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 37671 } } peers { permanent_uuid: "66c38532da714eceb1ef3488f4f5263d" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 38695 } }
I20260809 15:35:42.653606 5873 raft_consensus.cc:374] T 0285927261164c3581d7308de0043ffb P 31f3c6c99bab401fb6eb011dea8fb13b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 15:35:42.653681 5873 raft_consensus.cc:729] T 0285927261164c3581d7308de0043ffb P 31f3c6c99bab401fb6eb011dea8fb13b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 31f3c6c99bab401fb6eb011dea8fb13b, State: Initialized, Role: FOLLOWER
I20260809 15:35:42.654124 5873 consensus_queue.cc:260] T 0285927261164c3581d7308de0043ffb P 31f3c6c99bab401fb6eb011dea8fb13b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8ed96760f4584a5ca96e33c6ce9a8f2e" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41519 } } peers { permanent_uuid: "31f3c6c99bab401fb6eb011dea8fb13b" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 37671 } } peers { permanent_uuid: "66c38532da714eceb1ef3488f4f5263d" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 38695 } }
I20260809 15:35:42.654807 5689 heartbeater.cc:503] Master 127.3.89.254:33125 was elected leader, sending a full tablet report...
I20260809 15:35:42.655092 5873 ts_tablet_manager.cc:1428] T 0285927261164c3581d7308de0043ffb P 31f3c6c99bab401fb6eb011dea8fb13b: Time spent starting tablet: real 0.005s user 0.006s sys 0.000s
I20260809 15:35:42.700836 5875 raft_consensus.cc:482] T 0285927261164c3581d7308de0043ffb P 66c38532da714eceb1ef3488f4f5263d [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 15:35:42.701084 5875 raft_consensus.cc:504] T 0285927261164c3581d7308de0043ffb P 66c38532da714eceb1ef3488f4f5263d [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8ed96760f4584a5ca96e33c6ce9a8f2e" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41519 } } peers { permanent_uuid: "31f3c6c99bab401fb6eb011dea8fb13b" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 37671 } } peers { permanent_uuid: "66c38532da714eceb1ef3488f4f5263d" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 38695 } }
I20260809 15:35:42.702275 5875 leader_election.cc:288] T 0285927261164c3581d7308de0043ffb P 66c38532da714eceb1ef3488f4f5263d [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 8ed96760f4584a5ca96e33c6ce9a8f2e (127.3.89.195:41519), 31f3c6c99bab401fb6eb011dea8fb13b (127.3.89.194:37671)
I20260809 15:35:42.706337 5863 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:35:42.702646 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.89.193:33433 (local address 127.3.89.195:41519)
0809 15:35:42.702842 (+ 196us) server_negotiation.cc:206] Beginning negotiation
0809 15:35:42.702849 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 15:35:42.702980 (+ 131us) server_negotiation.cc:407] Connection header received
0809 15:35:42.703162 (+ 182us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 15:35:42.703166 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 15:35:42.703227 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 15:35:42.703321 (+ 94us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 15:35:42.703683 (+ 362us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:35:42.704320 (+ 637us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:35:42.705377 (+ 1057us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:35:42.705787 (+ 410us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:35:42.705915 (+ 128us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:35:42.705968 (+ 53us) server_negotiation.cc:1035] Waiting for connection context
0809 15:35:42.706145 (+ 177us) server_negotiation.cc:299] Negotiation successful
0809 15:35:42.706212 (+ 67us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":87}
I20260809 15:35:42.706353 5878 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:35:42.702510 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.89.195:41519 (local address 127.3.89.193:33433)
0809 15:35:42.702855 (+ 345us) negotiation.cc:107] Waiting for socket to connect
0809 15:35:42.702869 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0809 15:35:42.703035 (+ 166us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 15:35:42.703338 (+ 303us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 15:35:42.703341 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 15:35:42.703368 (+ 27us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 15:35:42.703543 (+ 175us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:35:42.703550 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:35:42.704434 (+ 884us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:35:42.704439 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:35:42.705251 (+ 812us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:35:42.705259 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:35:42.705908 (+ 649us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:35:42.705911 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:35:42.706010 (+ 99us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:35:42.706033 (+ 23us) client_negotiation.cc:769] Sending connection context
0809 15:35:42.706129 (+ 96us) client_negotiation.cc:240] Negotiation successful
0809 15:35:42.706199 (+ 70us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":274,"thread_start_us":132,"threads_started":1}
I20260809 15:35:42.706578 5867 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:35:42.703158 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.89.193:53101 (local address 127.3.89.194:37671)
0809 15:35:42.703291 (+ 133us) server_negotiation.cc:206] Beginning negotiation
0809 15:35:42.703297 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 15:35:42.703423 (+ 126us) server_negotiation.cc:407] Connection header received
0809 15:35:42.703537 (+ 114us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 15:35:42.703540 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 15:35:42.703598 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 15:35:42.703676 (+ 78us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 15:35:42.703981 (+ 305us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:35:42.704603 (+ 622us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:35:42.705602 (+ 999us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:35:42.706021 (+ 419us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:35:42.706149 (+ 128us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:35:42.706201 (+ 52us) server_negotiation.cc:1035] Waiting for connection context
0809 15:35:42.706397 (+ 196us) server_negotiation.cc:299] Negotiation successful
0809 15:35:42.706464 (+ 67us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":54}
I20260809 15:35:42.706609 5879 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:35:42.703017 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.89.194:37671 (local address 127.3.89.193:53101)
0809 15:35:42.703332 (+ 315us) negotiation.cc:107] Waiting for socket to connect
0809 15:35:42.703346 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0809 15:35:42.703449 (+ 103us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 15:35:42.703700 (+ 251us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 15:35:42.703702 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 15:35:42.703720 (+ 18us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 15:35:42.703863 (+ 143us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:35:42.703869 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:35:42.704726 (+ 857us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:35:42.704733 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:35:42.705475 (+ 742us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:35:42.705483 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:35:42.706142 (+ 659us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:35:42.706146 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:35:42.706259 (+ 113us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:35:42.706282 (+ 23us) client_negotiation.cc:769] Sending connection context
0809 15:35:42.706379 (+ 97us) client_negotiation.cc:240] Negotiation successful
0809 15:35:42.706450 (+ 71us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":242,"spinlock_wait_cycles":1920,"thread_start_us":107,"threads_started":1}
I20260809 15:35:42.707209 5795 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0285927261164c3581d7308de0043ffb" candidate_uuid: "66c38532da714eceb1ef3488f4f5263d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8ed96760f4584a5ca96e33c6ce9a8f2e" is_pre_election: true
I20260809 15:35:42.707365 5637 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0285927261164c3581d7308de0043ffb" candidate_uuid: "66c38532da714eceb1ef3488f4f5263d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "31f3c6c99bab401fb6eb011dea8fb13b" is_pre_election: true
I20260809 15:35:42.707504 5795 raft_consensus.cc:2454] T 0285927261164c3581d7308de0043ffb P 8ed96760f4584a5ca96e33c6ce9a8f2e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 66c38532da714eceb1ef3488f4f5263d in term 0.
I20260809 15:35:42.707686 5637 raft_consensus.cc:2454] T 0285927261164c3581d7308de0043ffb P 31f3c6c99bab401fb6eb011dea8fb13b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 66c38532da714eceb1ef3488f4f5263d in term 0.
I20260809 15:35:42.708022 5413 leader_election.cc:302] T 0285927261164c3581d7308de0043ffb P 66c38532da714eceb1ef3488f4f5263d [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 66c38532da714eceb1ef3488f4f5263d, 8ed96760f4584a5ca96e33c6ce9a8f2e; no voters:
I20260809 15:35:42.708317 5875 raft_consensus.cc:2781] T 0285927261164c3581d7308de0043ffb P 66c38532da714eceb1ef3488f4f5263d [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260809 15:35:42.708396 5875 raft_consensus.cc:482] T 0285927261164c3581d7308de0043ffb P 66c38532da714eceb1ef3488f4f5263d [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 15:35:42.708425 5875 raft_consensus.cc:3037] T 0285927261164c3581d7308de0043ffb P 66c38532da714eceb1ef3488f4f5263d [term 0 FOLLOWER]: Advancing to term 1
I20260809 15:35:42.709532 5875 raft_consensus.cc:504] T 0285927261164c3581d7308de0043ffb P 66c38532da714eceb1ef3488f4f5263d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8ed96760f4584a5ca96e33c6ce9a8f2e" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41519 } } peers { permanent_uuid: "31f3c6c99bab401fb6eb011dea8fb13b" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 37671 } } peers { permanent_uuid: "66c38532da714eceb1ef3488f4f5263d" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 38695 } }
I20260809 15:35:42.709883 5875 leader_election.cc:288] T 0285927261164c3581d7308de0043ffb P 66c38532da714eceb1ef3488f4f5263d [CANDIDATE]: Term 1 election: Requested vote from peers 8ed96760f4584a5ca96e33c6ce9a8f2e (127.3.89.195:41519), 31f3c6c99bab401fb6eb011dea8fb13b (127.3.89.194:37671)
I20260809 15:35:42.710202 5795 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0285927261164c3581d7308de0043ffb" candidate_uuid: "66c38532da714eceb1ef3488f4f5263d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8ed96760f4584a5ca96e33c6ce9a8f2e"
I20260809 15:35:42.710307 5637 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0285927261164c3581d7308de0043ffb" candidate_uuid: "66c38532da714eceb1ef3488f4f5263d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "31f3c6c99bab401fb6eb011dea8fb13b"
I20260809 15:35:42.710332 5795 raft_consensus.cc:3037] T 0285927261164c3581d7308de0043ffb P 8ed96760f4584a5ca96e33c6ce9a8f2e [term 0 FOLLOWER]: Advancing to term 1
I20260809 15:35:42.710469 5637 raft_consensus.cc:3037] T 0285927261164c3581d7308de0043ffb P 31f3c6c99bab401fb6eb011dea8fb13b [term 0 FOLLOWER]: Advancing to term 1
I20260809 15:35:42.711493 5795 raft_consensus.cc:2454] T 0285927261164c3581d7308de0043ffb P 8ed96760f4584a5ca96e33c6ce9a8f2e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 66c38532da714eceb1ef3488f4f5263d in term 1.
I20260809 15:35:42.711587 5637 raft_consensus.cc:2454] T 0285927261164c3581d7308de0043ffb P 31f3c6c99bab401fb6eb011dea8fb13b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 66c38532da714eceb1ef3488f4f5263d in term 1.
I20260809 15:35:42.711803 5413 leader_election.cc:302] T 0285927261164c3581d7308de0043ffb P 66c38532da714eceb1ef3488f4f5263d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 66c38532da714eceb1ef3488f4f5263d, 8ed96760f4584a5ca96e33c6ce9a8f2e; no voters:
I20260809 15:35:42.711979 5875 raft_consensus.cc:2781] T 0285927261164c3581d7308de0043ffb P 66c38532da714eceb1ef3488f4f5263d [term 1 FOLLOWER]: Leader election won for term 1
I20260809 15:35:42.712199 5875 raft_consensus.cc:686] T 0285927261164c3581d7308de0043ffb P 66c38532da714eceb1ef3488f4f5263d [term 1 LEADER]: Becoming Leader. State: Replica: 66c38532da714eceb1ef3488f4f5263d, State: Running, Role: LEADER
I20260809 15:35:42.712563 5875 consensus_queue.cc:237] T 0285927261164c3581d7308de0043ffb P 66c38532da714eceb1ef3488f4f5263d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8ed96760f4584a5ca96e33c6ce9a8f2e" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41519 } } peers { permanent_uuid: "31f3c6c99bab401fb6eb011dea8fb13b" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 37671 } } peers { permanent_uuid: "66c38532da714eceb1ef3488f4f5263d" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 38695 } }
I20260809 15:35:42.716006 5318 catalog_manager.cc:5495] T 0285927261164c3581d7308de0043ffb P 66c38532da714eceb1ef3488f4f5263d reported cstate change: term changed from 0 to 1, leader changed from <none> to 66c38532da714eceb1ef3488f4f5263d (127.3.89.193). New cstate: current_term: 1 leader_uuid: "66c38532da714eceb1ef3488f4f5263d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8ed96760f4584a5ca96e33c6ce9a8f2e" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41519 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "31f3c6c99bab401fb6eb011dea8fb13b" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 37671 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "66c38532da714eceb1ef3488f4f5263d" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 38695 } health_report { overall_health: HEALTHY } } }
Aug 09 15:35:42 dist-test-slave-2w3w krb5kdc[5270](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786289741, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.89.254@KRBTEST.COM for kudu/127.3.89.254@KRBTEST.COM
I20260809 15:35:42.774070 5370 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:35:42.760912 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45006 (local address 127.3.89.254:33125)
0809 15:35:42.761069 (+ 157us) server_negotiation.cc:206] Beginning negotiation
0809 15:35:42.761075 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 15:35:42.761270 (+ 195us) server_negotiation.cc:407] Connection header received
0809 15:35:42.761426 (+ 156us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 15:35:42.761429 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 15:35:42.761485 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 15:35:42.761573 (+ 88us) server_negotiation.cc:226] Negotiated authn=SASL
0809 15:35:42.762240 (+ 667us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:35:42.762765 (+ 525us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:35:42.763588 (+ 823us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:35:42.763804 (+ 216us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:35:42.763913 (+ 109us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:35:42.770692 (+ 6779us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 15:35:42.770695 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 15:35:42.770697 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 15:35:42.770716 (+ 19us) server_negotiation.cc:922] calling sasl_server_start()
0809 15:35:42.772257 (+ 1541us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 15:35:42.772772 (+ 515us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 15:35:42.772776 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 15:35:42.772778 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 15:35:42.772859 (+ 81us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 15:35:42.773135 (+ 276us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 15:35:42.773140 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 15:35:42.773142 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 15:35:42.773317 (+ 175us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 15:35:42.773426 (+ 109us) server_negotiation.cc:1035] Waiting for connection context
0809 15:35:42.773717 (+ 291us) server_negotiation.cc:299] Negotiation successful
0809 15:35:42.773912 (+ 195us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":94,"server-negotiator.queue_time_us":51}
I20260809 15:35:42.774049 5887 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:35:42.760857 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.89.254:33125 (local address 127.0.0.1:45006)
0809 15:35:42.761177 (+ 320us) negotiation.cc:107] Waiting for socket to connect
0809 15:35:42.761188 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0809 15:35:42.761327 (+ 139us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 15:35:42.761595 (+ 268us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 15:35:42.761598 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 15:35:42.762021 (+ 423us) client_negotiation.cc:189] Negotiated authn=SASL
0809 15:35:42.762112 (+ 91us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:35:42.762119 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:35:42.762876 (+ 757us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:35:42.762882 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:35:42.763469 (+ 587us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:35:42.763478 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:35:42.763915 (+ 437us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:35:42.763919 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:35:42.764010 (+ 91us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:35:42.769067 (+ 5057us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 15:35:42.769091 (+ 24us) client_negotiation.cc:656] calling sasl_client_start()
0809 15:35:42.770536 (+ 1445us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 15:35:42.772383 (+ 1847us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 15:35:42.772393 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 15:35:42.772399 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0809 15:35:42.772671 (+ 272us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 15:35:42.772950 (+ 279us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 15:35:42.772955 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 15:35:42.772963 (+ 8us) client_negotiation.cc:760] calling sasl_client_step()
0809 15:35:42.773046 (+ 83us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 15:35:42.773434 (+ 388us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 15:35:42.773444 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 15:35:42.773556 (+ 112us) client_negotiation.cc:769] Sending connection context
0809 15:35:42.773673 (+ 117us) client_negotiation.cc:240] Negotiation successful
0809 15:35:42.773839 (+ 166us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":244,"thread_start_us":120,"threads_started":1}
I20260809 15:35:42.778074 5318 catalog_manager.cc:1908] Servicing CreateTable request from {username='kudu', principal='kudu/127.3.89.254@KRBTEST.COM'} at 127.0.0.1:45006:
name: "kudu_system.kudu_transactions"
schema {
columns {
name: "txn_id"
type: INT64
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "entry_type"
type: INT8
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "identifier"
type: STRING
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "metadata"
type: STRING
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
rows: "<redacted>""\006\001\000\000\000\000\000\000\000\000\007\001@B\017\000\000\000\000\000"
indirect_data: "<redacted>"""
}
partition_schema {
range_schema {
columns {
name: "txn_id"
}
}
}
table_type: TXN_STATUS_TABLE
W20260809 15:35:42.778811 5318 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table kudu_system.kudu_transactions in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260809 15:35:42.783743 5465 tablet_service.cc:1467] Processing CreateTablet for tablet 41d3dd4df98b4df98efbe8d16d2a3e71 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=bf80100a5ddb49e3bf80363522ad5c00]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260809 15:35:42.784111 5615 tablet_service.cc:1467] Processing CreateTablet for tablet 41d3dd4df98b4df98efbe8d16d2a3e71 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=bf80100a5ddb49e3bf80363522ad5c00]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260809 15:35:42.784109 5773 tablet_service.cc:1467] Processing CreateTablet for tablet 41d3dd4df98b4df98efbe8d16d2a3e71 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=bf80100a5ddb49e3bf80363522ad5c00]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260809 15:35:42.784569 5465 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 41d3dd4df98b4df98efbe8d16d2a3e71. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 15:35:42.784699 5615 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 41d3dd4df98b4df98efbe8d16d2a3e71. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 15:35:42.784781 5773 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 41d3dd4df98b4df98efbe8d16d2a3e71. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 15:35:42.787827 5870 tablet_bootstrap.cc:492] T 41d3dd4df98b4df98efbe8d16d2a3e71 P 8ed96760f4584a5ca96e33c6ce9a8f2e: Bootstrap starting.
I20260809 15:35:42.787837 5869 tablet_bootstrap.cc:492] T 41d3dd4df98b4df98efbe8d16d2a3e71 P 66c38532da714eceb1ef3488f4f5263d: Bootstrap starting.
I20260809 15:35:42.787832 5873 tablet_bootstrap.cc:492] T 41d3dd4df98b4df98efbe8d16d2a3e71 P 31f3c6c99bab401fb6eb011dea8fb13b: Bootstrap starting.
I20260809 15:35:42.788633 5873 tablet_bootstrap.cc:654] T 41d3dd4df98b4df98efbe8d16d2a3e71 P 31f3c6c99bab401fb6eb011dea8fb13b: Neither blocks nor log segments found. Creating new log.
I20260809 15:35:42.788787 5869 tablet_bootstrap.cc:654] T 41d3dd4df98b4df98efbe8d16d2a3e71 P 66c38532da714eceb1ef3488f4f5263d: Neither blocks nor log segments found. Creating new log.
I20260809 15:35:42.788787 5870 tablet_bootstrap.cc:654] T 41d3dd4df98b4df98efbe8d16d2a3e71 P 8ed96760f4584a5ca96e33c6ce9a8f2e: Neither blocks nor log segments found. Creating new log.
I20260809 15:35:42.790077 5870 tablet_bootstrap.cc:492] T 41d3dd4df98b4df98efbe8d16d2a3e71 P 8ed96760f4584a5ca96e33c6ce9a8f2e: No bootstrap required, opened a new log
I20260809 15:35:42.790138 5873 tablet_bootstrap.cc:492] T 41d3dd4df98b4df98efbe8d16d2a3e71 P 31f3c6c99bab401fb6eb011dea8fb13b: No bootstrap required, opened a new log
I20260809 15:35:42.790159 5870 ts_tablet_manager.cc:1397] T 41d3dd4df98b4df98efbe8d16d2a3e71 P 8ed96760f4584a5ca96e33c6ce9a8f2e: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.002s
I20260809 15:35:42.790217 5873 ts_tablet_manager.cc:1397] T 41d3dd4df98b4df98efbe8d16d2a3e71 P 31f3c6c99bab401fb6eb011dea8fb13b: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260809 15:35:42.790619 5873 raft_consensus.cc:348] T 41d3dd4df98b4df98efbe8d16d2a3e71 P 31f3c6c99bab401fb6eb011dea8fb13b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "66c38532da714eceb1ef3488f4f5263d" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 38695 } } peers { permanent_uuid: "31f3c6c99bab401fb6eb011dea8fb13b" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 37671 } } peers { permanent_uuid: "8ed96760f4584a5ca96e33c6ce9a8f2e" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41519 } }
I20260809 15:35:42.790619 5870 raft_consensus.cc:348] T 41d3dd4df98b4df98efbe8d16d2a3e71 P 8ed96760f4584a5ca96e33c6ce9a8f2e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "66c38532da714eceb1ef3488f4f5263d" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 38695 } } peers { permanent_uuid: "31f3c6c99bab401fb6eb011dea8fb13b" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 37671 } } peers { permanent_uuid: "8ed96760f4584a5ca96e33c6ce9a8f2e" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41519 } }
I20260809 15:35:42.790723 5870 raft_consensus.cc:374] T 41d3dd4df98b4df98efbe8d16d2a3e71 P 8ed96760f4584a5ca96e33c6ce9a8f2e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 15:35:42.790723 5873 raft_consensus.cc:374] T 41d3dd4df98b4df98efbe8d16d2a3e71 P 31f3c6c99bab401fb6eb011dea8fb13b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 15:35:42.790740 5869 tablet_bootstrap.cc:492] T 41d3dd4df98b4df98efbe8d16d2a3e71 P 66c38532da714eceb1ef3488f4f5263d: No bootstrap required, opened a new log
I20260809 15:35:42.790757 5870 raft_consensus.cc:729] T 41d3dd4df98b4df98efbe8d16d2a3e71 P 8ed96760f4584a5ca96e33c6ce9a8f2e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8ed96760f4584a5ca96e33c6ce9a8f2e, State: Initialized, Role: FOLLOWER
I20260809 15:35:42.790757 5873 raft_consensus.cc:729] T 41d3dd4df98b4df98efbe8d16d2a3e71 P 31f3c6c99bab401fb6eb011dea8fb13b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 31f3c6c99bab401fb6eb011dea8fb13b, State: Initialized, Role: FOLLOWER
I20260809 15:35:42.790798 5869 ts_tablet_manager.cc:1397] T 41d3dd4df98b4df98efbe8d16d2a3e71 P 66c38532da714eceb1ef3488f4f5263d: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260809 15:35:42.790884 5873 consensus_queue.cc:260] T 41d3dd4df98b4df98efbe8d16d2a3e71 P 31f3c6c99bab401fb6eb011dea8fb13b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "66c38532da714eceb1ef3488f4f5263d" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 38695 } } peers { permanent_uuid: "31f3c6c99bab401fb6eb011dea8fb13b" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 37671 } } peers { permanent_uuid: "8ed96760f4584a5ca96e33c6ce9a8f2e" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41519 } }
I20260809 15:35:42.790884 5870 consensus_queue.cc:260] T 41d3dd4df98b4df98efbe8d16d2a3e71 P 8ed96760f4584a5ca96e33c6ce9a8f2e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "66c38532da714eceb1ef3488f4f5263d" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 38695 } } peers { permanent_uuid: "31f3c6c99bab401fb6eb011dea8fb13b" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 37671 } } peers { permanent_uuid: "8ed96760f4584a5ca96e33c6ce9a8f2e" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41519 } }
I20260809 15:35:42.791155 5876 tablet_replica.cc:439] T 41d3dd4df98b4df98efbe8d16d2a3e71 P 8ed96760f4584a5ca96e33c6ce9a8f2e: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "66c38532da714eceb1ef3488f4f5263d" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 38695 } } peers { permanent_uuid: "31f3c6c99bab401fb6eb011dea8fb13b" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 37671 } } peers { permanent_uuid: "8ed96760f4584a5ca96e33c6ce9a8f2e" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41519 } } }
I20260809 15:35:42.791316 5876 tablet_replica.cc:442] T 41d3dd4df98b4df98efbe8d16d2a3e71 P 8ed96760f4584a5ca96e33c6ce9a8f2e: This TxnStatusTablet replica's current role is: FOLLOWER
I20260809 15:35:42.791347 5873 ts_tablet_manager.cc:1428] T 41d3dd4df98b4df98efbe8d16d2a3e71 P 31f3c6c99bab401fb6eb011dea8fb13b: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260809 15:35:42.791303 5869 raft_consensus.cc:348] T 41d3dd4df98b4df98efbe8d16d2a3e71 P 66c38532da714eceb1ef3488f4f5263d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "66c38532da714eceb1ef3488f4f5263d" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 38695 } } peers { permanent_uuid: "31f3c6c99bab401fb6eb011dea8fb13b" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 37671 } } peers { permanent_uuid: "8ed96760f4584a5ca96e33c6ce9a8f2e" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41519 } }
I20260809 15:35:42.791411 5869 raft_consensus.cc:374] T 41d3dd4df98b4df98efbe8d16d2a3e71 P 66c38532da714eceb1ef3488f4f5263d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 15:35:42.791447 5869 raft_consensus.cc:729] T 41d3dd4df98b4df98efbe8d16d2a3e71 P 66c38532da714eceb1ef3488f4f5263d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 66c38532da714eceb1ef3488f4f5263d, State: Initialized, Role: FOLLOWER
I20260809 15:35:42.791553 5869 consensus_queue.cc:260] T 41d3dd4df98b4df98efbe8d16d2a3e71 P 66c38532da714eceb1ef3488f4f5263d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "66c38532da714eceb1ef3488f4f5263d" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 38695 } } peers { permanent_uuid: "31f3c6c99bab401fb6eb011dea8fb13b" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 37671 } } peers { permanent_uuid: "8ed96760f4584a5ca96e33c6ce9a8f2e" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41519 } }
I20260809 15:35:42.791687 5877 tablet_replica.cc:439] T 41d3dd4df98b4df98efbe8d16d2a3e71 P 31f3c6c99bab401fb6eb011dea8fb13b: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "66c38532da714eceb1ef3488f4f5263d" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 38695 } } peers { permanent_uuid: "31f3c6c99bab401fb6eb011dea8fb13b" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 37671 } } peers { permanent_uuid: "8ed96760f4584a5ca96e33c6ce9a8f2e" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41519 } } }
I20260809 15:35:42.791818 5877 tablet_replica.cc:442] T 41d3dd4df98b4df98efbe8d16d2a3e71 P 31f3c6c99bab401fb6eb011dea8fb13b: This TxnStatusTablet replica's current role is: FOLLOWER
I20260809 15:35:42.791821 5869 ts_tablet_manager.cc:1428] T 41d3dd4df98b4df98efbe8d16d2a3e71 P 66c38532da714eceb1ef3488f4f5263d: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260809 15:35:42.792050 5875 tablet_replica.cc:439] T 41d3dd4df98b4df98efbe8d16d2a3e71 P 66c38532da714eceb1ef3488f4f5263d: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "66c38532da714eceb1ef3488f4f5263d" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 38695 } } peers { permanent_uuid: "31f3c6c99bab401fb6eb011dea8fb13b" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 37671 } } peers { permanent_uuid: "8ed96760f4584a5ca96e33c6ce9a8f2e" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41519 } } }
I20260809 15:35:42.792157 5875 tablet_replica.cc:442] T 41d3dd4df98b4df98efbe8d16d2a3e71 P 66c38532da714eceb1ef3488f4f5263d: This TxnStatusTablet replica's current role is: FOLLOWER
I20260809 15:35:42.791177 5870 ts_tablet_manager.cc:1428] T 41d3dd4df98b4df98efbe8d16d2a3e71 P 8ed96760f4584a5ca96e33c6ce9a8f2e: Time spent starting tablet: real 0.001s user 0.001s sys 0.001s
W20260809 15:35:42.821164 5843 tablet.cc:2369] T 0285927261164c3581d7308de0043ffb P 8ed96760f4584a5ca96e33c6ce9a8f2e: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260809 15:35:42.832419 5690 tablet.cc:2369] T 0285927261164c3581d7308de0043ffb P 31f3c6c99bab401fb6eb011dea8fb13b: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260809 15:35:42.855479 5876 raft_consensus.cc:482] T 41d3dd4df98b4df98efbe8d16d2a3e71 P 8ed96760f4584a5ca96e33c6ce9a8f2e [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 15:35:42.855604 5876 raft_consensus.cc:504] T 41d3dd4df98b4df98efbe8d16d2a3e71 P 8ed96760f4584a5ca96e33c6ce9a8f2e [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "66c38532da714eceb1ef3488f4f5263d" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 38695 } } peers { permanent_uuid: "31f3c6c99bab401fb6eb011dea8fb13b" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 37671 } } peers { permanent_uuid: "8ed96760f4584a5ca96e33c6ce9a8f2e" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41519 } }
I20260809 15:35:42.856613 5876 leader_election.cc:288] T 41d3dd4df98b4df98efbe8d16d2a3e71 P 8ed96760f4584a5ca96e33c6ce9a8f2e [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 66c38532da714eceb1ef3488f4f5263d (127.3.89.193:38695), 31f3c6c99bab401fb6eb011dea8fb13b (127.3.89.194:37671)
W20260809 15:35:42.859661 5537 tablet.cc:2369] T 0285927261164c3581d7308de0043ffb P 66c38532da714eceb1ef3488f4f5263d: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260809 15:35:42.861056 5866 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:35:42.856989 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.89.195:48253 (local address 127.3.89.193:38695)
0809 15:35:42.857168 (+ 179us) server_negotiation.cc:206] Beginning negotiation
0809 15:35:42.857174 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 15:35:42.857190 (+ 16us) server_negotiation.cc:407] Connection header received
0809 15:35:42.857248 (+ 58us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 15:35:42.857254 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 15:35:42.857331 (+ 77us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 15:35:42.857413 (+ 82us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 15:35:42.857721 (+ 308us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:35:42.858493 (+ 772us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:35:42.860027 (+ 1534us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:35:42.860499 (+ 472us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:35:42.860632 (+ 133us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:35:42.860684 (+ 52us) server_negotiation.cc:1035] Waiting for connection context
0809 15:35:42.860866 (+ 182us) server_negotiation.cc:299] Negotiation successful
0809 15:35:42.860940 (+ 74us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":58}
I20260809 15:35:42.861056 5725 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:35:42.856839 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.89.193:38695 (local address 127.3.89.195:48253)
0809 15:35:42.856947 (+ 108us) negotiation.cc:107] Waiting for socket to connect
0809 15:35:42.856981 (+ 34us) client_negotiation.cc:174] Beginning negotiation
0809 15:35:42.857083 (+ 102us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 15:35:42.857446 (+ 363us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 15:35:42.857449 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 15:35:42.857469 (+ 20us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 15:35:42.857603 (+ 134us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:35:42.857613 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:35:42.858607 (+ 994us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:35:42.858610 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:35:42.859884 (+ 1274us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:35:42.859912 (+ 28us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:35:42.860626 (+ 714us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:35:42.860630 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:35:42.860732 (+ 102us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:35:42.860753 (+ 21us) client_negotiation.cc:769] Sending connection context
0809 15:35:42.860849 (+ 96us) client_negotiation.cc:240] Negotiation successful
0809 15:35:42.860920 (+ 71us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":31}
I20260809 15:35:42.861186 5867 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:35:42.857127 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.89.195:60167 (local address 127.3.89.194:37671)
0809 15:35:42.857276 (+ 149us) server_negotiation.cc:206] Beginning negotiation
0809 15:35:42.857279 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0809 15:35:42.857639 (+ 360us) server_negotiation.cc:407] Connection header received
0809 15:35:42.857737 (+ 98us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 15:35:42.857740 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 15:35:42.857787 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 15:35:42.857863 (+ 76us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 15:35:42.858102 (+ 239us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:35:42.858752 (+ 650us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:35:42.860172 (+ 1420us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:35:42.860586 (+ 414us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:35:42.860720 (+ 134us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:35:42.860771 (+ 51us) server_negotiation.cc:1035] Waiting for connection context
0809 15:35:42.860979 (+ 208us) server_negotiation.cc:299] Negotiation successful
0809 15:35:42.861054 (+ 75us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":49}
I20260809 15:35:42.861234 5891 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:35:42.856839 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.89.194:37671 (local address 127.3.89.195:60167)
0809 15:35:42.857535 (+ 696us) negotiation.cc:107] Waiting for socket to connect
0809 15:35:42.857550 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0809 15:35:42.857653 (+ 103us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 15:35:42.857876 (+ 223us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 15:35:42.857878 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 15:35:42.857891 (+ 13us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 15:35:42.857996 (+ 105us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:35:42.858000 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:35:42.858852 (+ 852us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:35:42.858855 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:35:42.860060 (+ 1205us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:35:42.860067 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:35:42.860734 (+ 667us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:35:42.860738 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:35:42.860857 (+ 119us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:35:42.860874 (+ 17us) client_negotiation.cc:769] Sending connection context
0809 15:35:42.860961 (+ 87us) client_negotiation.cc:240] Negotiation successful
0809 15:35:42.861026 (+ 65us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":639,"mutex_wait_us":337,"thread_start_us":128,"threads_started":1}
I20260809 15:35:42.861676 5488 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "41d3dd4df98b4df98efbe8d16d2a3e71" candidate_uuid: "8ed96760f4584a5ca96e33c6ce9a8f2e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "66c38532da714eceb1ef3488f4f5263d" is_pre_election: true
I20260809 15:35:42.861732 5637 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "41d3dd4df98b4df98efbe8d16d2a3e71" candidate_uuid: "8ed96760f4584a5ca96e33c6ce9a8f2e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "31f3c6c99bab401fb6eb011dea8fb13b" is_pre_election: true
I20260809 15:35:42.861903 5637 raft_consensus.cc:2454] T 41d3dd4df98b4df98efbe8d16d2a3e71 P 31f3c6c99bab401fb6eb011dea8fb13b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8ed96760f4584a5ca96e33c6ce9a8f2e in term 0.
I20260809 15:35:42.861902 5488 raft_consensus.cc:2454] T 41d3dd4df98b4df98efbe8d16d2a3e71 P 66c38532da714eceb1ef3488f4f5263d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8ed96760f4584a5ca96e33c6ce9a8f2e in term 0.
I20260809 15:35:42.862277 5720 leader_election.cc:302] T 41d3dd4df98b4df98efbe8d16d2a3e71 P 8ed96760f4584a5ca96e33c6ce9a8f2e [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 31f3c6c99bab401fb6eb011dea8fb13b, 8ed96760f4584a5ca96e33c6ce9a8f2e; no voters:
I20260809 15:35:42.862572 5876 raft_consensus.cc:2781] T 41d3dd4df98b4df98efbe8d16d2a3e71 P 8ed96760f4584a5ca96e33c6ce9a8f2e [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260809 15:35:42.862648 5876 raft_consensus.cc:482] T 41d3dd4df98b4df98efbe8d16d2a3e71 P 8ed96760f4584a5ca96e33c6ce9a8f2e [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 15:35:42.862704 5876 raft_consensus.cc:3037] T 41d3dd4df98b4df98efbe8d16d2a3e71 P 8ed96760f4584a5ca96e33c6ce9a8f2e [term 0 FOLLOWER]: Advancing to term 1
I20260809 15:35:42.863695 5876 raft_consensus.cc:504] T 41d3dd4df98b4df98efbe8d16d2a3e71 P 8ed96760f4584a5ca96e33c6ce9a8f2e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "66c38532da714eceb1ef3488f4f5263d" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 38695 } } peers { permanent_uuid: "31f3c6c99bab401fb6eb011dea8fb13b" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 37671 } } peers { permanent_uuid: "8ed96760f4584a5ca96e33c6ce9a8f2e" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41519 } }
I20260809 15:35:42.864037 5876 leader_election.cc:288] T 41d3dd4df98b4df98efbe8d16d2a3e71 P 8ed96760f4584a5ca96e33c6ce9a8f2e [CANDIDATE]: Term 1 election: Requested vote from peers 66c38532da714eceb1ef3488f4f5263d (127.3.89.193:38695), 31f3c6c99bab401fb6eb011dea8fb13b (127.3.89.194:37671)
I20260809 15:35:42.864409 5637 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "41d3dd4df98b4df98efbe8d16d2a3e71" candidate_uuid: "8ed96760f4584a5ca96e33c6ce9a8f2e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "31f3c6c99bab401fb6eb011dea8fb13b"
I20260809 15:35:42.864408 5488 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "41d3dd4df98b4df98efbe8d16d2a3e71" candidate_uuid: "8ed96760f4584a5ca96e33c6ce9a8f2e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "66c38532da714eceb1ef3488f4f5263d"
I20260809 15:35:42.864533 5637 raft_consensus.cc:3037] T 41d3dd4df98b4df98efbe8d16d2a3e71 P 31f3c6c99bab401fb6eb011dea8fb13b [term 0 FOLLOWER]: Advancing to term 1
I20260809 15:35:42.864547 5488 raft_consensus.cc:3037] T 41d3dd4df98b4df98efbe8d16d2a3e71 P 66c38532da714eceb1ef3488f4f5263d [term 0 FOLLOWER]: Advancing to term 1
I20260809 15:35:42.865538 5637 raft_consensus.cc:2454] T 41d3dd4df98b4df98efbe8d16d2a3e71 P 31f3c6c99bab401fb6eb011dea8fb13b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8ed96760f4584a5ca96e33c6ce9a8f2e in term 1.
I20260809 15:35:42.865741 5488 raft_consensus.cc:2454] T 41d3dd4df98b4df98efbe8d16d2a3e71 P 66c38532da714eceb1ef3488f4f5263d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8ed96760f4584a5ca96e33c6ce9a8f2e in term 1.
I20260809 15:35:42.866026 5720 leader_election.cc:302] T 41d3dd4df98b4df98efbe8d16d2a3e71 P 8ed96760f4584a5ca96e33c6ce9a8f2e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 31f3c6c99bab401fb6eb011dea8fb13b, 8ed96760f4584a5ca96e33c6ce9a8f2e; no voters:
I20260809 15:35:42.866170 5876 raft_consensus.cc:2781] T 41d3dd4df98b4df98efbe8d16d2a3e71 P 8ed96760f4584a5ca96e33c6ce9a8f2e [term 1 FOLLOWER]: Leader election won for term 1
I20260809 15:35:42.866364 5876 raft_consensus.cc:686] T 41d3dd4df98b4df98efbe8d16d2a3e71 P 8ed96760f4584a5ca96e33c6ce9a8f2e [term 1 LEADER]: Becoming Leader. State: Replica: 8ed96760f4584a5ca96e33c6ce9a8f2e, State: Running, Role: LEADER
I20260809 15:35:42.866667 5876 consensus_queue.cc:237] T 41d3dd4df98b4df98efbe8d16d2a3e71 P 8ed96760f4584a5ca96e33c6ce9a8f2e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "66c38532da714eceb1ef3488f4f5263d" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 38695 } } peers { permanent_uuid: "31f3c6c99bab401fb6eb011dea8fb13b" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 37671 } } peers { permanent_uuid: "8ed96760f4584a5ca96e33c6ce9a8f2e" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41519 } }
I20260809 15:35:42.868983 5892 tablet_replica.cc:439] T 41d3dd4df98b4df98efbe8d16d2a3e71 P 8ed96760f4584a5ca96e33c6ce9a8f2e: TxnStatusTablet state changed. Reason: New leader 8ed96760f4584a5ca96e33c6ce9a8f2e. Latest consensus state: current_term: 1 leader_uuid: "8ed96760f4584a5ca96e33c6ce9a8f2e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "66c38532da714eceb1ef3488f4f5263d" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 38695 } } peers { permanent_uuid: "31f3c6c99bab401fb6eb011dea8fb13b" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 37671 } } peers { permanent_uuid: "8ed96760f4584a5ca96e33c6ce9a8f2e" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41519 } } }
I20260809 15:35:42.869100 5892 tablet_replica.cc:442] T 41d3dd4df98b4df98efbe8d16d2a3e71 P 8ed96760f4584a5ca96e33c6ce9a8f2e: This TxnStatusTablet replica's current role is: LEADER
I20260809 15:35:42.869364 5895 txn_status_manager.cc:873] Waiting until node catch up with all replicated operations in previous term...
I20260809 15:35:42.869578 5895 txn_status_manager.cc:929] Loading transaction status metadata into memory...
I20260809 15:35:42.869956 5318 catalog_manager.cc:5495] T 41d3dd4df98b4df98efbe8d16d2a3e71 P 8ed96760f4584a5ca96e33c6ce9a8f2e reported cstate change: term changed from 0 to 1, leader changed from <none> to 8ed96760f4584a5ca96e33c6ce9a8f2e (127.3.89.195). New cstate: current_term: 1 leader_uuid: "8ed96760f4584a5ca96e33c6ce9a8f2e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "66c38532da714eceb1ef3488f4f5263d" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 38695 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "31f3c6c99bab401fb6eb011dea8fb13b" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 37671 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8ed96760f4584a5ca96e33c6ce9a8f2e" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41519 } health_report { overall_health: HEALTHY } } }
I20260809 15:35:42.893766 5863 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:35:42.889997 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46748 (local address 127.3.89.195:41519)
0809 15:35:42.890150 (+ 153us) server_negotiation.cc:206] Beginning negotiation
0809 15:35:42.890155 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 15:35:42.890170 (+ 15us) server_negotiation.cc:407] Connection header received
0809 15:35:42.890343 (+ 173us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 15:35:42.890346 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 15:35:42.890407 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 15:35:42.890522 (+ 115us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 15:35:42.890798 (+ 276us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:35:42.891372 (+ 574us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:35:42.892213 (+ 841us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:35:42.892448 (+ 235us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:35:42.892585 (+ 137us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:35:42.892910 (+ 325us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 15:35:42.893339 (+ 429us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 15:35:42.893466 (+ 127us) server_negotiation.cc:1035] Waiting for connection context
0809 15:35:42.893546 (+ 80us) server_negotiation.cc:299] Negotiation successful
0809 15:35:42.893599 (+ 53us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":67}
I20260809 15:35:42.893741 5887 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:35:42.889851 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.89.195:41519 (local address 127.0.0.1:46748)
0809 15:35:42.890024 (+ 173us) negotiation.cc:107] Waiting for socket to connect
0809 15:35:42.890059 (+ 35us) client_negotiation.cc:174] Beginning negotiation
0809 15:35:42.890173 (+ 114us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 15:35:42.890536 (+ 363us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 15:35:42.890539 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 15:35:42.890546 (+ 7us) client_negotiation.cc:189] Negotiated authn=TOKEN
0809 15:35:42.890683 (+ 137us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:35:42.890688 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:35:42.891486 (+ 798us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:35:42.891491 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:35:42.892090 (+ 599us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:35:42.892098 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:35:42.892586 (+ 488us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:35:42.892589 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:35:42.892715 (+ 126us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:35:42.892757 (+ 42us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0809 15:35:42.893443 (+ 686us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0809 15:35:42.893460 (+ 17us) client_negotiation.cc:769] Sending connection context
0809 15:35:42.893527 (+ 67us) client_negotiation.cc:240] Negotiation successful
0809 15:35:42.893611 (+ 84us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":58}
I20260809 15:35:42.899154 5637 raft_consensus.cc:1261] T 41d3dd4df98b4df98efbe8d16d2a3e71 P 31f3c6c99bab401fb6eb011dea8fb13b [term 1 FOLLOWER]: Refusing update from remote peer 8ed96760f4584a5ca96e33c6ce9a8f2e: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 15:35:42.899173 5488 raft_consensus.cc:1261] T 41d3dd4df98b4df98efbe8d16d2a3e71 P 66c38532da714eceb1ef3488f4f5263d [term 1 FOLLOWER]: Refusing update from remote peer 8ed96760f4584a5ca96e33c6ce9a8f2e: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 15:35:42.899825 5892 consensus_queue.cc:1035] T 41d3dd4df98b4df98efbe8d16d2a3e71 P 8ed96760f4584a5ca96e33c6ce9a8f2e [LEADER]: Connected to new peer: Peer: permanent_uuid: "66c38532da714eceb1ef3488f4f5263d" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 38695 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 15:35:42.900089 5876 consensus_queue.cc:1035] T 41d3dd4df98b4df98efbe8d16d2a3e71 P 8ed96760f4584a5ca96e33c6ce9a8f2e [LEADER]: Connected to new peer: Peer: permanent_uuid: "31f3c6c99bab401fb6eb011dea8fb13b" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 37671 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 15:35:42.903362 5875 tablet_replica.cc:439] T 41d3dd4df98b4df98efbe8d16d2a3e71 P 66c38532da714eceb1ef3488f4f5263d: TxnStatusTablet state changed. Reason: New leader 8ed96760f4584a5ca96e33c6ce9a8f2e. Latest consensus state: current_term: 1 leader_uuid: "8ed96760f4584a5ca96e33c6ce9a8f2e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "66c38532da714eceb1ef3488f4f5263d" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 38695 } } peers { permanent_uuid: "31f3c6c99bab401fb6eb011dea8fb13b" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 37671 } } peers { permanent_uuid: "8ed96760f4584a5ca96e33c6ce9a8f2e" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41519 } } }
I20260809 15:35:42.903486 5875 tablet_replica.cc:442] T 41d3dd4df98b4df98efbe8d16d2a3e71 P 66c38532da714eceb1ef3488f4f5263d: This TxnStatusTablet replica's current role is: FOLLOWER
I20260809 15:35:42.903983 5877 tablet_replica.cc:439] T 41d3dd4df98b4df98efbe8d16d2a3e71 P 31f3c6c99bab401fb6eb011dea8fb13b: TxnStatusTablet state changed. Reason: New leader 8ed96760f4584a5ca96e33c6ce9a8f2e. Latest consensus state: current_term: 1 leader_uuid: "8ed96760f4584a5ca96e33c6ce9a8f2e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "66c38532da714eceb1ef3488f4f5263d" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 38695 } } peers { permanent_uuid: "31f3c6c99bab401fb6eb011dea8fb13b" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 37671 } } peers { permanent_uuid: "8ed96760f4584a5ca96e33c6ce9a8f2e" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41519 } } }
I20260809 15:35:42.904093 5877 tablet_replica.cc:442] T 41d3dd4df98b4df98efbe8d16d2a3e71 P 31f3c6c99bab401fb6eb011dea8fb13b: This TxnStatusTablet replica's current role is: FOLLOWER
I20260809 15:35:42.908265 5876 tablet_replica.cc:439] T 41d3dd4df98b4df98efbe8d16d2a3e71 P 8ed96760f4584a5ca96e33c6ce9a8f2e: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "8ed96760f4584a5ca96e33c6ce9a8f2e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "66c38532da714eceb1ef3488f4f5263d" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 38695 } } peers { permanent_uuid: "31f3c6c99bab401fb6eb011dea8fb13b" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 37671 } } peers { permanent_uuid: "8ed96760f4584a5ca96e33c6ce9a8f2e" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41519 } } }
I20260809 15:35:42.908403 5876 tablet_replica.cc:442] T 41d3dd4df98b4df98efbe8d16d2a3e71 P 8ed96760f4584a5ca96e33c6ce9a8f2e: This TxnStatusTablet replica's current role is: LEADER
I20260809 15:35:42.908437 5892 tablet_replica.cc:439] T 41d3dd4df98b4df98efbe8d16d2a3e71 P 8ed96760f4584a5ca96e33c6ce9a8f2e: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "8ed96760f4584a5ca96e33c6ce9a8f2e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "66c38532da714eceb1ef3488f4f5263d" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 38695 } } peers { permanent_uuid: "31f3c6c99bab401fb6eb011dea8fb13b" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 37671 } } peers { permanent_uuid: "8ed96760f4584a5ca96e33c6ce9a8f2e" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41519 } } }
I20260809 15:35:42.908560 5892 tablet_replica.cc:442] T 41d3dd4df98b4df98efbe8d16d2a3e71 P 8ed96760f4584a5ca96e33c6ce9a8f2e: This TxnStatusTablet replica's current role is: LEADER
I20260809 15:35:42.909672 5875 tablet_replica.cc:439] T 41d3dd4df98b4df98efbe8d16d2a3e71 P 66c38532da714eceb1ef3488f4f5263d: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "8ed96760f4584a5ca96e33c6ce9a8f2e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "66c38532da714eceb1ef3488f4f5263d" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 38695 } } peers { permanent_uuid: "31f3c6c99bab401fb6eb011dea8fb13b" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 37671 } } peers { permanent_uuid: "8ed96760f4584a5ca96e33c6ce9a8f2e" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41519 } } }
I20260809 15:35:42.909781 5875 tablet_replica.cc:442] T 41d3dd4df98b4df98efbe8d16d2a3e71 P 66c38532da714eceb1ef3488f4f5263d: This TxnStatusTablet replica's current role is: FOLLOWER
I20260809 15:35:42.910801 5877 tablet_replica.cc:439] T 41d3dd4df98b4df98efbe8d16d2a3e71 P 31f3c6c99bab401fb6eb011dea8fb13b: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "8ed96760f4584a5ca96e33c6ce9a8f2e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "66c38532da714eceb1ef3488f4f5263d" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 38695 } } peers { permanent_uuid: "31f3c6c99bab401fb6eb011dea8fb13b" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 37671 } } peers { permanent_uuid: "8ed96760f4584a5ca96e33c6ce9a8f2e" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41519 } } }
I20260809 15:35:42.910916 5877 tablet_replica.cc:442] T 41d3dd4df98b4df98efbe8d16d2a3e71 P 31f3c6c99bab401fb6eb011dea8fb13b: This TxnStatusTablet replica's current role is: FOLLOWER
I20260809 15:35:42.917702 5866 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:35:42.913899 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54918 (local address 127.3.89.193:38695)
0809 15:35:42.914009 (+ 110us) server_negotiation.cc:206] Beginning negotiation
0809 15:35:42.914014 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 15:35:42.914132 (+ 118us) server_negotiation.cc:407] Connection header received
0809 15:35:42.914312 (+ 180us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 15:35:42.914315 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 15:35:42.914375 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 15:35:42.914476 (+ 101us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 15:35:42.914713 (+ 237us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:35:42.915397 (+ 684us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:35:42.916258 (+ 861us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:35:42.916507 (+ 249us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:35:42.916627 (+ 120us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:35:42.916916 (+ 289us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 15:35:42.917320 (+ 404us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 15:35:42.917442 (+ 122us) server_negotiation.cc:1035] Waiting for connection context
0809 15:35:42.917513 (+ 71us) server_negotiation.cc:299] Negotiation successful
0809 15:35:42.917567 (+ 54us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":37}
I20260809 15:35:42.928182 5913 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:35:42.924659 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.89.195:41519 (local address 127.3.89.193:36591)
0809 15:35:42.924940 (+ 281us) negotiation.cc:107] Waiting for socket to connect
0809 15:35:42.924952 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0809 15:35:42.925028 (+ 76us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 15:35:42.925259 (+ 231us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 15:35:42.925260 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 15:35:42.925266 (+ 6us) client_negotiation.cc:189] Negotiated authn=TOKEN
0809 15:35:42.925362 (+ 96us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:35:42.925366 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:35:42.926315 (+ 949us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:35:42.926317 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:35:42.926932 (+ 615us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:35:42.926940 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:35:42.927513 (+ 573us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:35:42.927515 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:35:42.927583 (+ 68us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:35:42.927620 (+ 37us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0809 15:35:42.927934 (+ 314us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0809 15:35:42.927938 (+ 4us) client_negotiation.cc:769] Sending connection context
0809 15:35:42.927989 (+ 51us) client_negotiation.cc:240] Negotiation successful
0809 15:35:42.928049 (+ 60us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":181,"thread_start_us":116,"threads_started":1}
I20260809 15:35:42.928198 5863 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:35:42.924772 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.89.193:36591 (local address 127.3.89.195:41519)
0809 15:35:42.924888 (+ 116us) server_negotiation.cc:206] Beginning negotiation
0809 15:35:42.924892 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 15:35:42.925020 (+ 128us) server_negotiation.cc:407] Connection header received
0809 15:35:42.925121 (+ 101us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 15:35:42.925124 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 15:35:42.925173 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 15:35:42.925244 (+ 71us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 15:35:42.925467 (+ 223us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:35:42.926204 (+ 737us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:35:42.927072 (+ 868us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:35:42.927411 (+ 339us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:35:42.927516 (+ 105us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:35:42.927757 (+ 241us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 15:35:42.927852 (+ 95us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 15:35:42.927923 (+ 71us) server_negotiation.cc:1035] Waiting for connection context
0809 15:35:42.928014 (+ 91us) server_negotiation.cc:299] Negotiation successful
0809 15:35:42.928063 (+ 49us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":49}
I20260809 15:35:42.935015 5637 raft_consensus.cc:1261] T 0285927261164c3581d7308de0043ffb P 31f3c6c99bab401fb6eb011dea8fb13b [term 1 FOLLOWER]: Refusing update from remote peer 66c38532da714eceb1ef3488f4f5263d: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 15:35:42.935173 5795 raft_consensus.cc:1261] T 0285927261164c3581d7308de0043ffb P 8ed96760f4584a5ca96e33c6ce9a8f2e [term 1 FOLLOWER]: Refusing update from remote peer 66c38532da714eceb1ef3488f4f5263d: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 15:35:42.935510 5880 consensus_queue.cc:1035] T 0285927261164c3581d7308de0043ffb P 66c38532da714eceb1ef3488f4f5263d [LEADER]: Connected to new peer: Peer: permanent_uuid: "31f3c6c99bab401fb6eb011dea8fb13b" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 37671 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 15:35:42.935693 5875 consensus_queue.cc:1035] T 0285927261164c3581d7308de0043ffb P 66c38532da714eceb1ef3488f4f5263d [LEADER]: Connected to new peer: Peer: permanent_uuid: "8ed96760f4584a5ca96e33c6ce9a8f2e" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41519 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 15:35:42.941792 5900 mvcc.cc:205] Tried to move back new op lower bound from 7316642787057229824 to 7316642786153684992. Current Snapshot: MvccSnapshot[applied={T|T < 7316642787057229824}]
I20260809 15:35:42.942749 5898 mvcc.cc:205] Tried to move back new op lower bound from 7316642787057229824 to 7316642786153684992. Current Snapshot: MvccSnapshot[applied={T|T < 7316642787057229824}]
I20260809 15:35:42.960767 5850 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:35:42.957163 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.89.193:38695 (local address 127.3.89.195:56353)
0809 15:35:42.957310 (+ 147us) negotiation.cc:107] Waiting for socket to connect
0809 15:35:42.957333 (+ 23us) client_negotiation.cc:174] Beginning negotiation
0809 15:35:42.957407 (+ 74us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 15:35:42.957622 (+ 215us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 15:35:42.957624 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 15:35:42.957629 (+ 5us) client_negotiation.cc:189] Negotiated authn=TOKEN
0809 15:35:42.957728 (+ 99us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:35:42.957732 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:35:42.958734 (+ 1002us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:35:42.958737 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:35:42.959428 (+ 691us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:35:42.959438 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:35:42.960016 (+ 578us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:35:42.960019 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:35:42.960115 (+ 96us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:35:42.960154 (+ 39us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0809 15:35:42.960467 (+ 313us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0809 15:35:42.960476 (+ 9us) client_negotiation.cc:769] Sending connection context
0809 15:35:42.960564 (+ 88us) client_negotiation.cc:240] Negotiation successful
0809 15:35:42.960641 (+ 77us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":27}
I20260809 15:35:42.960767 5866 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:35:42.957230 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.89.195:56353 (local address 127.3.89.193:38695)
0809 15:35:42.957338 (+ 108us) server_negotiation.cc:206] Beginning negotiation
0809 15:35:42.957343 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 15:35:42.957376 (+ 33us) server_negotiation.cc:407] Connection header received
0809 15:35:42.957478 (+ 102us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 15:35:42.957482 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 15:35:42.957529 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 15:35:42.957610 (+ 81us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 15:35:42.957836 (+ 226us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:35:42.958614 (+ 778us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:35:42.959552 (+ 938us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:35:42.959890 (+ 338us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:35:42.960020 (+ 130us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:35:42.960291 (+ 271us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 15:35:42.960367 (+ 76us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 15:35:42.960463 (+ 96us) server_negotiation.cc:1035] Waiting for connection context
0809 15:35:42.960578 (+ 115us) server_negotiation.cc:299] Negotiation successful
0809 15:35:42.960633 (+ 55us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":45}
I20260809 15:35:42.984508 5318 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:44998:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 15:35:42.984725 5318 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:44998:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 15:35:42.987704 5318 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 726056f89ec1474594dd4b44dcfadbfe: Sending DeleteTablet for 3 replicas of tablet 0285927261164c3581d7308de0043ffb
I20260809 15:35:42.988699 5465 tablet_service.cc:1514] Processing DeleteTablet for tablet 0285927261164c3581d7308de0043ffb with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 15:35:42 UTC) from {username='kudu', principal='kudu/127.3.89.254@KRBTEST.COM'} at 127.0.0.1:54908
I20260809 15:35:42.988749 5615 tablet_service.cc:1514] Processing DeleteTablet for tablet 0285927261164c3581d7308de0043ffb with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 15:35:42 UTC) from {username='kudu', principal='kudu/127.3.89.254@KRBTEST.COM'} at 127.0.0.1:42122
I20260809 15:35:42.989387 5773 tablet_service.cc:1514] Processing DeleteTablet for tablet 0285927261164c3581d7308de0043ffb with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 15:35:42 UTC) from {username='kudu', principal='kudu/127.3.89.254@KRBTEST.COM'} at 127.0.0.1:46744
I20260809 15:35:42.989938 5924 tablet_replica.cc:330] T 0285927261164c3581d7308de0043ffb P 8ed96760f4584a5ca96e33c6ce9a8f2e: stopping tablet replica
I20260809 15:35:42.989893 5318 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:44998:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260809 15:35:42.990332 5923 tablet_replica.cc:330] T 0285927261164c3581d7308de0043ffb P 31f3c6c99bab401fb6eb011dea8fb13b: stopping tablet replica
W20260809 15:35:42.990330 5318 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260809 15:35:42.990718 5923 raft_consensus.cc:2229] T 0285927261164c3581d7308de0043ffb P 31f3c6c99bab401fb6eb011dea8fb13b [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 15:35:42.991261 5922 tablet_replica.cc:330] T 0285927261164c3581d7308de0043ffb P 66c38532da714eceb1ef3488f4f5263d: stopping tablet replica
I20260809 15:35:42.991531 5922 raft_consensus.cc:2229] T 0285927261164c3581d7308de0043ffb P 66c38532da714eceb1ef3488f4f5263d [term 1 LEADER]: Raft consensus shutting down.
I20260809 15:35:42.991878 5922 raft_consensus.cc:2258] T 0285927261164c3581d7308de0043ffb P 66c38532da714eceb1ef3488f4f5263d [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 15:35:42.992357 5924 raft_consensus.cc:2229] T 0285927261164c3581d7308de0043ffb P 8ed96760f4584a5ca96e33c6ce9a8f2e [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 15:35:42.992695 5923 raft_consensus.cc:2258] T 0285927261164c3581d7308de0043ffb P 31f3c6c99bab401fb6eb011dea8fb13b [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 15:35:42.992913 5924 raft_consensus.cc:2258] T 0285927261164c3581d7308de0043ffb P 8ed96760f4584a5ca96e33c6ce9a8f2e [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 15:35:42.993698 5922 ts_tablet_manager.cc:1905] T 0285927261164c3581d7308de0043ffb P 66c38532da714eceb1ef3488f4f5263d: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 15:35:42.995829 5465 tablet_service.cc:1467] Processing CreateTablet for tablet e95aa6581e02425f9ab49f69dbc6939c (DEFAULT_TABLE table=test-table [id=8529331515d34b6cae64660ee2cac958]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 15:35:42.995865 5615 tablet_service.cc:1467] Processing CreateTablet for tablet e95aa6581e02425f9ab49f69dbc6939c (DEFAULT_TABLE table=test-table [id=8529331515d34b6cae64660ee2cac958]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 15:35:42.996160 5615 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e95aa6581e02425f9ab49f69dbc6939c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 15:35:42.996188 5465 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e95aa6581e02425f9ab49f69dbc6939c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 15:35:42.997000 5922 ts_tablet_manager.cc:1918] T 0285927261164c3581d7308de0043ffb P 66c38532da714eceb1ef3488f4f5263d: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260809 15:35:42.997081 5922 log.cc:1196] T 0285927261164c3581d7308de0043ffb P 66c38532da714eceb1ef3488f4f5263d: Deleting WAL directory at /tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786289732823074-3431-0/minicluster-data/ts-0/wal/wals/0285927261164c3581d7308de0043ffb
I20260809 15:35:42.997427 5922 ts_tablet_manager.cc:1939] T 0285927261164c3581d7308de0043ffb P 66c38532da714eceb1ef3488f4f5263d: Deleting consensus metadata
I20260809 15:35:42.998674 5869 tablet_bootstrap.cc:492] T e95aa6581e02425f9ab49f69dbc6939c P 66c38532da714eceb1ef3488f4f5263d: Bootstrap starting.
I20260809 15:35:42.998687 5873 tablet_bootstrap.cc:492] T e95aa6581e02425f9ab49f69dbc6939c P 31f3c6c99bab401fb6eb011dea8fb13b: Bootstrap starting.
I20260809 15:35:42.999648 5869 tablet_bootstrap.cc:654] T e95aa6581e02425f9ab49f69dbc6939c P 66c38532da714eceb1ef3488f4f5263d: Neither blocks nor log segments found. Creating new log.
I20260809 15:35:42.999680 5873 tablet_bootstrap.cc:654] T e95aa6581e02425f9ab49f69dbc6939c P 31f3c6c99bab401fb6eb011dea8fb13b: Neither blocks nor log segments found. Creating new log.
I20260809 15:35:43.000415 5773 tablet_service.cc:1467] Processing CreateTablet for tablet e95aa6581e02425f9ab49f69dbc6939c (DEFAULT_TABLE table=test-table [id=8529331515d34b6cae64660ee2cac958]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 15:35:43.000677 5773 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e95aa6581e02425f9ab49f69dbc6939c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 15:35:43.000862 5873 tablet_bootstrap.cc:492] T e95aa6581e02425f9ab49f69dbc6939c P 31f3c6c99bab401fb6eb011dea8fb13b: No bootstrap required, opened a new log
I20260809 15:35:43.000938 5873 ts_tablet_manager.cc:1397] T e95aa6581e02425f9ab49f69dbc6939c P 31f3c6c99bab401fb6eb011dea8fb13b: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260809 15:35:43.001411 5873 raft_consensus.cc:348] T e95aa6581e02425f9ab49f69dbc6939c P 31f3c6c99bab401fb6eb011dea8fb13b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "66c38532da714eceb1ef3488f4f5263d" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 38695 } } peers { permanent_uuid: "31f3c6c99bab401fb6eb011dea8fb13b" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 37671 } } peers { permanent_uuid: "8ed96760f4584a5ca96e33c6ce9a8f2e" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41519 } }
I20260809 15:35:43.001524 5873 raft_consensus.cc:374] T e95aa6581e02425f9ab49f69dbc6939c P 31f3c6c99bab401fb6eb011dea8fb13b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 15:35:43.001555 5873 raft_consensus.cc:729] T e95aa6581e02425f9ab49f69dbc6939c P 31f3c6c99bab401fb6eb011dea8fb13b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 31f3c6c99bab401fb6eb011dea8fb13b, State: Initialized, Role: FOLLOWER
I20260809 15:35:43.001660 5873 consensus_queue.cc:260] T e95aa6581e02425f9ab49f69dbc6939c P 31f3c6c99bab401fb6eb011dea8fb13b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "66c38532da714eceb1ef3488f4f5263d" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 38695 } } peers { permanent_uuid: "31f3c6c99bab401fb6eb011dea8fb13b" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 37671 } } peers { permanent_uuid: "8ed96760f4584a5ca96e33c6ce9a8f2e" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41519 } }
I20260809 15:35:43.001922 5873 ts_tablet_manager.cc:1428] T e95aa6581e02425f9ab49f69dbc6939c P 31f3c6c99bab401fb6eb011dea8fb13b: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260809 15:35:43.003782 5303 catalog_manager.cc:4841] TS 66c38532da714eceb1ef3488f4f5263d (127.3.89.193:38695): tablet 0285927261164c3581d7308de0043ffb (table test-table [id=04331610cf50454694403002c6b87934]) successfully deleted
I20260809 15:35:43.005079 5870 tablet_bootstrap.cc:492] T e95aa6581e02425f9ab49f69dbc6939c P 8ed96760f4584a5ca96e33c6ce9a8f2e: Bootstrap starting.
I20260809 15:35:43.006106 5870 tablet_bootstrap.cc:654] T e95aa6581e02425f9ab49f69dbc6939c P 8ed96760f4584a5ca96e33c6ce9a8f2e: Neither blocks nor log segments found. Creating new log.
I20260809 15:35:43.007953 5870 tablet_bootstrap.cc:492] T e95aa6581e02425f9ab49f69dbc6939c P 8ed96760f4584a5ca96e33c6ce9a8f2e: No bootstrap required, opened a new log
I20260809 15:35:43.008041 5870 ts_tablet_manager.cc:1397] T e95aa6581e02425f9ab49f69dbc6939c P 8ed96760f4584a5ca96e33c6ce9a8f2e: Time spent bootstrapping tablet: real 0.003s user 0.001s sys 0.000s
I20260809 15:35:43.008333 5924 ts_tablet_manager.cc:1905] T 0285927261164c3581d7308de0043ffb P 8ed96760f4584a5ca96e33c6ce9a8f2e: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 15:35:43.009325 5923 ts_tablet_manager.cc:1905] T 0285927261164c3581d7308de0043ffb P 31f3c6c99bab401fb6eb011dea8fb13b: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 15:35:43.008502 5870 raft_consensus.cc:348] T e95aa6581e02425f9ab49f69dbc6939c P 8ed96760f4584a5ca96e33c6ce9a8f2e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "66c38532da714eceb1ef3488f4f5263d" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 38695 } } peers { permanent_uuid: "31f3c6c99bab401fb6eb011dea8fb13b" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 37671 } } peers { permanent_uuid: "8ed96760f4584a5ca96e33c6ce9a8f2e" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41519 } }
I20260809 15:35:43.011073 5870 raft_consensus.cc:374] T e95aa6581e02425f9ab49f69dbc6939c P 8ed96760f4584a5ca96e33c6ce9a8f2e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 15:35:43.011133 5870 raft_consensus.cc:729] T e95aa6581e02425f9ab49f69dbc6939c P 8ed96760f4584a5ca96e33c6ce9a8f2e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8ed96760f4584a5ca96e33c6ce9a8f2e, State: Initialized, Role: FOLLOWER
I20260809 15:35:43.011278 5870 consensus_queue.cc:260] T e95aa6581e02425f9ab49f69dbc6939c P 8ed96760f4584a5ca96e33c6ce9a8f2e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "66c38532da714eceb1ef3488f4f5263d" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 38695 } } peers { permanent_uuid: "31f3c6c99bab401fb6eb011dea8fb13b" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 37671 } } peers { permanent_uuid: "8ed96760f4584a5ca96e33c6ce9a8f2e" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41519 } }
I20260809 15:35:43.011562 5870 ts_tablet_manager.cc:1428] T e95aa6581e02425f9ab49f69dbc6939c P 8ed96760f4584a5ca96e33c6ce9a8f2e: Time spent starting tablet: real 0.003s user 0.001s sys 0.000s
I20260809 15:35:43.012835 5923 ts_tablet_manager.cc:1918] T 0285927261164c3581d7308de0043ffb P 31f3c6c99bab401fb6eb011dea8fb13b: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260809 15:35:43.013006 5923 log.cc:1196] T 0285927261164c3581d7308de0043ffb P 31f3c6c99bab401fb6eb011dea8fb13b: Deleting WAL directory at /tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786289732823074-3431-0/minicluster-data/ts-1/wal/wals/0285927261164c3581d7308de0043ffb
I20260809 15:35:43.013410 5923 ts_tablet_manager.cc:1939] T 0285927261164c3581d7308de0043ffb P 31f3c6c99bab401fb6eb011dea8fb13b: Deleting consensus metadata
I20260809 15:35:43.015095 5869 tablet_bootstrap.cc:492] T e95aa6581e02425f9ab49f69dbc6939c P 66c38532da714eceb1ef3488f4f5263d: No bootstrap required, opened a new log
I20260809 15:35:43.015204 5869 ts_tablet_manager.cc:1397] T e95aa6581e02425f9ab49f69dbc6939c P 66c38532da714eceb1ef3488f4f5263d: Time spent bootstrapping tablet: real 0.017s user 0.002s sys 0.000s
I20260809 15:35:43.015681 5869 raft_consensus.cc:348] T e95aa6581e02425f9ab49f69dbc6939c P 66c38532da714eceb1ef3488f4f5263d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "66c38532da714eceb1ef3488f4f5263d" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 38695 } } peers { permanent_uuid: "31f3c6c99bab401fb6eb011dea8fb13b" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 37671 } } peers { permanent_uuid: "8ed96760f4584a5ca96e33c6ce9a8f2e" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41519 } }
I20260809 15:35:43.015805 5869 raft_consensus.cc:374] T e95aa6581e02425f9ab49f69dbc6939c P 66c38532da714eceb1ef3488f4f5263d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 15:35:43.015802 5924 ts_tablet_manager.cc:1918] T 0285927261164c3581d7308de0043ffb P 8ed96760f4584a5ca96e33c6ce9a8f2e: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260809 15:35:43.015846 5869 raft_consensus.cc:729] T e95aa6581e02425f9ab49f69dbc6939c P 66c38532da714eceb1ef3488f4f5263d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 66c38532da714eceb1ef3488f4f5263d, State: Initialized, Role: FOLLOWER
I20260809 15:35:43.015888 5924 log.cc:1196] T 0285927261164c3581d7308de0043ffb P 8ed96760f4584a5ca96e33c6ce9a8f2e: Deleting WAL directory at /tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786289732823074-3431-0/minicluster-data/ts-2/wal/wals/0285927261164c3581d7308de0043ffb
I20260809 15:35:43.015993 5869 consensus_queue.cc:260] T e95aa6581e02425f9ab49f69dbc6939c P 66c38532da714eceb1ef3488f4f5263d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "66c38532da714eceb1ef3488f4f5263d" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 38695 } } peers { permanent_uuid: "31f3c6c99bab401fb6eb011dea8fb13b" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 37671 } } peers { permanent_uuid: "8ed96760f4584a5ca96e33c6ce9a8f2e" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41519 } }
I20260809 15:35:43.016201 5924 ts_tablet_manager.cc:1939] T 0285927261164c3581d7308de0043ffb P 8ed96760f4584a5ca96e33c6ce9a8f2e: Deleting consensus metadata
I20260809 15:35:43.016302 5869 ts_tablet_manager.cc:1428] T e95aa6581e02425f9ab49f69dbc6939c P 66c38532da714eceb1ef3488f4f5263d: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260809 15:35:43.017671 5305 catalog_manager.cc:4841] TS 8ed96760f4584a5ca96e33c6ce9a8f2e (127.3.89.195:41519): tablet 0285927261164c3581d7308de0043ffb (table test-table [id=04331610cf50454694403002c6b87934]) successfully deleted
I20260809 15:35:43.018383 5305 catalog_manager.cc:4841] TS 31f3c6c99bab401fb6eb011dea8fb13b (127.3.89.194:37671): tablet 0285927261164c3581d7308de0043ffb (table test-table [id=04331610cf50454694403002c6b87934]) successfully deleted
I20260809 15:35:43.042831 5917 raft_consensus.cc:482] T e95aa6581e02425f9ab49f69dbc6939c P 66c38532da714eceb1ef3488f4f5263d [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 15:35:43.043008 5917 raft_consensus.cc:504] T e95aa6581e02425f9ab49f69dbc6939c P 66c38532da714eceb1ef3488f4f5263d [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "66c38532da714eceb1ef3488f4f5263d" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 38695 } } peers { permanent_uuid: "31f3c6c99bab401fb6eb011dea8fb13b" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 37671 } } peers { permanent_uuid: "8ed96760f4584a5ca96e33c6ce9a8f2e" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41519 } }
I20260809 15:35:43.043433 5917 leader_election.cc:288] T e95aa6581e02425f9ab49f69dbc6939c P 66c38532da714eceb1ef3488f4f5263d [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 31f3c6c99bab401fb6eb011dea8fb13b (127.3.89.194:37671), 8ed96760f4584a5ca96e33c6ce9a8f2e (127.3.89.195:41519)
I20260809 15:35:43.043844 5637 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e95aa6581e02425f9ab49f69dbc6939c" candidate_uuid: "66c38532da714eceb1ef3488f4f5263d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "31f3c6c99bab401fb6eb011dea8fb13b" is_pre_election: true
I20260809 15:35:43.043852 5795 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e95aa6581e02425f9ab49f69dbc6939c" candidate_uuid: "66c38532da714eceb1ef3488f4f5263d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8ed96760f4584a5ca96e33c6ce9a8f2e" is_pre_election: true
I20260809 15:35:43.043979 5637 raft_consensus.cc:2454] T e95aa6581e02425f9ab49f69dbc6939c P 31f3c6c99bab401fb6eb011dea8fb13b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 66c38532da714eceb1ef3488f4f5263d in term 0.
I20260809 15:35:43.044008 5795 raft_consensus.cc:2454] T e95aa6581e02425f9ab49f69dbc6939c P 8ed96760f4584a5ca96e33c6ce9a8f2e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 66c38532da714eceb1ef3488f4f5263d in term 0.
I20260809 15:35:43.044314 5413 leader_election.cc:302] T e95aa6581e02425f9ab49f69dbc6939c P 66c38532da714eceb1ef3488f4f5263d [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 31f3c6c99bab401fb6eb011dea8fb13b, 66c38532da714eceb1ef3488f4f5263d; no voters:
I20260809 15:35:43.044581 5917 raft_consensus.cc:2781] T e95aa6581e02425f9ab49f69dbc6939c P 66c38532da714eceb1ef3488f4f5263d [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260809 15:35:43.044628 5917 raft_consensus.cc:482] T e95aa6581e02425f9ab49f69dbc6939c P 66c38532da714eceb1ef3488f4f5263d [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 15:35:43.044663 5917 raft_consensus.cc:3037] T e95aa6581e02425f9ab49f69dbc6939c P 66c38532da714eceb1ef3488f4f5263d [term 0 FOLLOWER]: Advancing to term 1
I20260809 15:35:43.046463 5917 raft_consensus.cc:504] T e95aa6581e02425f9ab49f69dbc6939c P 66c38532da714eceb1ef3488f4f5263d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "66c38532da714eceb1ef3488f4f5263d" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 38695 } } peers { permanent_uuid: "31f3c6c99bab401fb6eb011dea8fb13b" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 37671 } } peers { permanent_uuid: "8ed96760f4584a5ca96e33c6ce9a8f2e" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41519 } }
I20260809 15:35:43.046806 5917 leader_election.cc:288] T e95aa6581e02425f9ab49f69dbc6939c P 66c38532da714eceb1ef3488f4f5263d [CANDIDATE]: Term 1 election: Requested vote from peers 31f3c6c99bab401fb6eb011dea8fb13b (127.3.89.194:37671), 8ed96760f4584a5ca96e33c6ce9a8f2e (127.3.89.195:41519)
I20260809 15:35:43.047307 5637 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e95aa6581e02425f9ab49f69dbc6939c" candidate_uuid: "66c38532da714eceb1ef3488f4f5263d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "31f3c6c99bab401fb6eb011dea8fb13b"
I20260809 15:35:43.047427 5637 raft_consensus.cc:3037] T e95aa6581e02425f9ab49f69dbc6939c P 31f3c6c99bab401fb6eb011dea8fb13b [term 0 FOLLOWER]: Advancing to term 1
I20260809 15:35:43.048539 5637 raft_consensus.cc:2454] T e95aa6581e02425f9ab49f69dbc6939c P 31f3c6c99bab401fb6eb011dea8fb13b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 66c38532da714eceb1ef3488f4f5263d in term 1.
I20260809 15:35:43.048882 5413 leader_election.cc:302] T e95aa6581e02425f9ab49f69dbc6939c P 66c38532da714eceb1ef3488f4f5263d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 31f3c6c99bab401fb6eb011dea8fb13b, 66c38532da714eceb1ef3488f4f5263d; no voters:
I20260809 15:35:43.049027 5917 raft_consensus.cc:2781] T e95aa6581e02425f9ab49f69dbc6939c P 66c38532da714eceb1ef3488f4f5263d [term 1 FOLLOWER]: Leader election won for term 1
I20260809 15:35:43.049358 5917 raft_consensus.cc:686] T e95aa6581e02425f9ab49f69dbc6939c P 66c38532da714eceb1ef3488f4f5263d [term 1 LEADER]: Becoming Leader. State: Replica: 66c38532da714eceb1ef3488f4f5263d, State: Running, Role: LEADER
I20260809 15:35:43.049497 5917 consensus_queue.cc:237] T e95aa6581e02425f9ab49f69dbc6939c P 66c38532da714eceb1ef3488f4f5263d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "66c38532da714eceb1ef3488f4f5263d" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 38695 } } peers { permanent_uuid: "31f3c6c99bab401fb6eb011dea8fb13b" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 37671 } } peers { permanent_uuid: "8ed96760f4584a5ca96e33c6ce9a8f2e" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41519 } }
I20260809 15:35:43.050942 5795 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e95aa6581e02425f9ab49f69dbc6939c" candidate_uuid: "66c38532da714eceb1ef3488f4f5263d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8ed96760f4584a5ca96e33c6ce9a8f2e"
I20260809 15:35:43.051096 5318 catalog_manager.cc:5495] T e95aa6581e02425f9ab49f69dbc6939c P 66c38532da714eceb1ef3488f4f5263d reported cstate change: term changed from 0 to 1, leader changed from <none> to 66c38532da714eceb1ef3488f4f5263d (127.3.89.193). New cstate: current_term: 1 leader_uuid: "66c38532da714eceb1ef3488f4f5263d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "66c38532da714eceb1ef3488f4f5263d" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 38695 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "31f3c6c99bab401fb6eb011dea8fb13b" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 37671 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8ed96760f4584a5ca96e33c6ce9a8f2e" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41519 } health_report { overall_health: UNKNOWN } } }
I20260809 15:35:43.051280 5795 raft_consensus.cc:3037] T e95aa6581e02425f9ab49f69dbc6939c P 8ed96760f4584a5ca96e33c6ce9a8f2e [term 0 FOLLOWER]: Advancing to term 1
I20260809 15:35:43.053717 5795 raft_consensus.cc:2454] T e95aa6581e02425f9ab49f69dbc6939c P 8ed96760f4584a5ca96e33c6ce9a8f2e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 66c38532da714eceb1ef3488f4f5263d in term 1.
I20260809 15:35:43.081554 5795 raft_consensus.cc:1261] T e95aa6581e02425f9ab49f69dbc6939c P 8ed96760f4584a5ca96e33c6ce9a8f2e [term 1 FOLLOWER]: Refusing update from remote peer 66c38532da714eceb1ef3488f4f5263d: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 15:35:43.081905 5637 raft_consensus.cc:1261] T e95aa6581e02425f9ab49f69dbc6939c P 31f3c6c99bab401fb6eb011dea8fb13b [term 1 FOLLOWER]: Refusing update from remote peer 66c38532da714eceb1ef3488f4f5263d: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 15:35:43.081959 5917 consensus_queue.cc:1035] T e95aa6581e02425f9ab49f69dbc6939c P 66c38532da714eceb1ef3488f4f5263d [LEADER]: Connected to new peer: Peer: permanent_uuid: "8ed96760f4584a5ca96e33c6ce9a8f2e" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41519 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 15:35:43.082255 5917 consensus_queue.cc:1035] T e95aa6581e02425f9ab49f69dbc6939c P 66c38532da714eceb1ef3488f4f5263d [LEADER]: Connected to new peer: Peer: permanent_uuid: "31f3c6c99bab401fb6eb011dea8fb13b" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 37671 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 15:35:43.084757 5899 mvcc.cc:205] Tried to move back new op lower bound from 7316642787657936896 to 7316642787532038144. Current Snapshot: MvccSnapshot[applied={T|T < 7316642787657936896}]
W20260809 15:35:43.097126 5899 tablet_replica.cc:1298] Aborted: operation has been aborted: cancelling pending write operations
I20260809 15:35:44.648350 5316 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:44998:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 15:35:44.648509 5316 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:44998:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 15:35:44.650902 5316 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 726056f89ec1474594dd4b44dcfadbfe: Sending DeleteTablet for 3 replicas of tablet e95aa6581e02425f9ab49f69dbc6939c
I20260809 15:35:44.651481 5465 tablet_service.cc:1514] Processing DeleteTablet for tablet e95aa6581e02425f9ab49f69dbc6939c with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 15:35:44 UTC) from {username='kudu', principal='kudu/127.3.89.254@KRBTEST.COM'} at 127.0.0.1:54908
I20260809 15:35:44.651607 5773 tablet_service.cc:1514] Processing DeleteTablet for tablet e95aa6581e02425f9ab49f69dbc6939c with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 15:35:44 UTC) from {username='kudu', principal='kudu/127.3.89.254@KRBTEST.COM'} at 127.0.0.1:46744
I20260809 15:35:44.651875 5952 tablet_replica.cc:330] T e95aa6581e02425f9ab49f69dbc6939c P 66c38532da714eceb1ef3488f4f5263d: stopping tablet replica
I20260809 15:35:44.651984 5953 tablet_replica.cc:330] T e95aa6581e02425f9ab49f69dbc6939c P 8ed96760f4584a5ca96e33c6ce9a8f2e: stopping tablet replica
I20260809 15:35:44.652045 5952 raft_consensus.cc:2229] T e95aa6581e02425f9ab49f69dbc6939c P 66c38532da714eceb1ef3488f4f5263d [term 1 LEADER]: Raft consensus shutting down.
I20260809 15:35:44.652149 5953 raft_consensus.cc:2229] T e95aa6581e02425f9ab49f69dbc6939c P 8ed96760f4584a5ca96e33c6ce9a8f2e [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 15:35:44.652257 5952 raft_consensus.cc:2258] T e95aa6581e02425f9ab49f69dbc6939c P 66c38532da714eceb1ef3488f4f5263d [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 15:35:44.652285 5953 raft_consensus.cc:2258] T e95aa6581e02425f9ab49f69dbc6939c P 8ed96760f4584a5ca96e33c6ce9a8f2e [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 15:35:44.652912 5952 ts_tablet_manager.cc:1905] T e95aa6581e02425f9ab49f69dbc6939c P 66c38532da714eceb1ef3488f4f5263d: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 15:35:44.653021 5953 ts_tablet_manager.cc:1905] T e95aa6581e02425f9ab49f69dbc6939c P 8ed96760f4584a5ca96e33c6ce9a8f2e: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 15:35:44.654160 5615 tablet_service.cc:1514] Processing DeleteTablet for tablet e95aa6581e02425f9ab49f69dbc6939c with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 15:35:44 UTC) from {username='kudu', principal='kudu/127.3.89.254@KRBTEST.COM'} at 127.0.0.1:42122
I20260809 15:35:44.654546 5955 tablet_replica.cc:330] T e95aa6581e02425f9ab49f69dbc6939c P 31f3c6c99bab401fb6eb011dea8fb13b: stopping tablet replica
I20260809 15:35:44.654712 5955 raft_consensus.cc:2229] T e95aa6581e02425f9ab49f69dbc6939c P 31f3c6c99bab401fb6eb011dea8fb13b [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 15:35:44.654876 5955 raft_consensus.cc:2258] T e95aa6581e02425f9ab49f69dbc6939c P 31f3c6c99bab401fb6eb011dea8fb13b [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 15:35:44.655536 5952 ts_tablet_manager.cc:1918] T e95aa6581e02425f9ab49f69dbc6939c P 66c38532da714eceb1ef3488f4f5263d: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260809 15:35:44.655624 5952 log.cc:1196] T e95aa6581e02425f9ab49f69dbc6939c P 66c38532da714eceb1ef3488f4f5263d: Deleting WAL directory at /tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786289732823074-3431-0/minicluster-data/ts-0/wal/wals/e95aa6581e02425f9ab49f69dbc6939c
I20260809 15:35:44.655650 5955 ts_tablet_manager.cc:1905] T e95aa6581e02425f9ab49f69dbc6939c P 31f3c6c99bab401fb6eb011dea8fb13b: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 15:35:44.655939 5952 ts_tablet_manager.cc:1939] T e95aa6581e02425f9ab49f69dbc6939c P 66c38532da714eceb1ef3488f4f5263d: Deleting consensus metadata
I20260809 15:35:44.656570 5953 ts_tablet_manager.cc:1918] T e95aa6581e02425f9ab49f69dbc6939c P 8ed96760f4584a5ca96e33c6ce9a8f2e: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260809 15:35:44.656626 5303 catalog_manager.cc:4841] TS 66c38532da714eceb1ef3488f4f5263d (127.3.89.193:38695): tablet e95aa6581e02425f9ab49f69dbc6939c (table test-table [id=8529331515d34b6cae64660ee2cac958]) successfully deleted
I20260809 15:35:44.656669 5953 log.cc:1196] T e95aa6581e02425f9ab49f69dbc6939c P 8ed96760f4584a5ca96e33c6ce9a8f2e: Deleting WAL directory at /tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786289732823074-3431-0/minicluster-data/ts-2/wal/wals/e95aa6581e02425f9ab49f69dbc6939c
I20260809 15:35:44.657006 5953 ts_tablet_manager.cc:1939] T e95aa6581e02425f9ab49f69dbc6939c P 8ed96760f4584a5ca96e33c6ce9a8f2e: Deleting consensus metadata
I20260809 15:35:44.657807 5305 catalog_manager.cc:4841] TS 8ed96760f4584a5ca96e33c6ce9a8f2e (127.3.89.195:41519): tablet e95aa6581e02425f9ab49f69dbc6939c (table test-table [id=8529331515d34b6cae64660ee2cac958]) successfully deleted
I20260809 15:35:44.658380 5955 ts_tablet_manager.cc:1918] T e95aa6581e02425f9ab49f69dbc6939c P 31f3c6c99bab401fb6eb011dea8fb13b: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260809 15:35:44.658483 5955 log.cc:1196] T e95aa6581e02425f9ab49f69dbc6939c P 31f3c6c99bab401fb6eb011dea8fb13b: Deleting WAL directory at /tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786289732823074-3431-0/minicluster-data/ts-1/wal/wals/e95aa6581e02425f9ab49f69dbc6939c
I20260809 15:35:44.658736 5955 ts_tablet_manager.cc:1939] T e95aa6581e02425f9ab49f69dbc6939c P 31f3c6c99bab401fb6eb011dea8fb13b: Deleting consensus metadata
I20260809 15:35:44.659405 5305 catalog_manager.cc:4841] TS 31f3c6c99bab401fb6eb011dea8fb13b (127.3.89.194:37671): tablet e95aa6581e02425f9ab49f69dbc6939c (table test-table [id=8529331515d34b6cae64660ee2cac958]) successfully deleted
Aug 09 15:35:44 dist-test-slave-2w3w krb5kdc[5270](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786289744, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Aug 09 15:35:44 dist-test-slave-2w3w krb5kdc[5270](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786289744, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.3.89.254@KRBTEST.COM
I20260809 15:35:44.683291 5962 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:35:44.674786 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45020 (local address 127.3.89.254:33125)
0809 15:35:44.675085 (+ 299us) server_negotiation.cc:206] Beginning negotiation
0809 15:35:44.675092 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 15:35:44.675133 (+ 41us) server_negotiation.cc:407] Connection header received
0809 15:35:44.675270 (+ 137us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 15:35:44.675273 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 15:35:44.675331 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 15:35:44.675438 (+ 107us) server_negotiation.cc:226] Negotiated authn=SASL
0809 15:35:44.676135 (+ 697us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:35:44.676673 (+ 538us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:35:44.677494 (+ 821us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:35:44.677714 (+ 220us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:35:44.677790 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:35:44.680284 (+ 2494us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 15:35:44.680287 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 15:35:44.680289 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 15:35:44.680313 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0809 15:35:44.681723 (+ 1410us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 15:35:44.682099 (+ 376us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 15:35:44.682102 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 15:35:44.682103 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 15:35:44.682182 (+ 79us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 15:35:44.682414 (+ 232us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 15:35:44.682416 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 15:35:44.682417 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 15:35:44.682588 (+ 171us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 15:35:44.682691 (+ 103us) server_negotiation.cc:1035] Waiting for connection context
0809 15:35:44.682892 (+ 201us) server_negotiation.cc:299] Negotiation successful
0809 15:35:44.683080 (+ 188us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":230,"thread_start_us":112,"threads_started":1}
I20260809 15:35:44.685858 5316 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:45020:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260809 15:35:44.686260 5316 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260809 15:35:44.691012 5615 tablet_service.cc:1467] Processing CreateTablet for tablet 544cf040ec1d46d8be4c46ca22189909 (DEFAULT_TABLE table=test-table [id=6a7cb370114e408badacc86ec7eb2ae7]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 15:35:44.691146 5773 tablet_service.cc:1467] Processing CreateTablet for tablet 544cf040ec1d46d8be4c46ca22189909 (DEFAULT_TABLE table=test-table [id=6a7cb370114e408badacc86ec7eb2ae7]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 15:35:44.691356 5615 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 544cf040ec1d46d8be4c46ca22189909. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 15:35:44.691480 5773 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 544cf040ec1d46d8be4c46ca22189909. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 15:35:44.691223 5465 tablet_service.cc:1467] Processing CreateTablet for tablet 544cf040ec1d46d8be4c46ca22189909 (DEFAULT_TABLE table=test-table [id=6a7cb370114e408badacc86ec7eb2ae7]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 15:35:44.691730 5465 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 544cf040ec1d46d8be4c46ca22189909. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 15:35:44.694340 5964 tablet_bootstrap.cc:492] T 544cf040ec1d46d8be4c46ca22189909 P 8ed96760f4584a5ca96e33c6ce9a8f2e: Bootstrap starting.
I20260809 15:35:44.694510 5963 tablet_bootstrap.cc:492] T 544cf040ec1d46d8be4c46ca22189909 P 31f3c6c99bab401fb6eb011dea8fb13b: Bootstrap starting.
I20260809 15:35:44.694752 5965 tablet_bootstrap.cc:492] T 544cf040ec1d46d8be4c46ca22189909 P 66c38532da714eceb1ef3488f4f5263d: Bootstrap starting.
I20260809 15:35:44.695233 5964 tablet_bootstrap.cc:654] T 544cf040ec1d46d8be4c46ca22189909 P 8ed96760f4584a5ca96e33c6ce9a8f2e: Neither blocks nor log segments found. Creating new log.
I20260809 15:35:44.695576 5963 tablet_bootstrap.cc:654] T 544cf040ec1d46d8be4c46ca22189909 P 31f3c6c99bab401fb6eb011dea8fb13b: Neither blocks nor log segments found. Creating new log.
I20260809 15:35:44.695811 5965 tablet_bootstrap.cc:654] T 544cf040ec1d46d8be4c46ca22189909 P 66c38532da714eceb1ef3488f4f5263d: Neither blocks nor log segments found. Creating new log.
I20260809 15:35:44.696609 5964 tablet_bootstrap.cc:492] T 544cf040ec1d46d8be4c46ca22189909 P 8ed96760f4584a5ca96e33c6ce9a8f2e: No bootstrap required, opened a new log
I20260809 15:35:44.696753 5964 ts_tablet_manager.cc:1397] T 544cf040ec1d46d8be4c46ca22189909 P 8ed96760f4584a5ca96e33c6ce9a8f2e: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260809 15:35:44.696892 5963 tablet_bootstrap.cc:492] T 544cf040ec1d46d8be4c46ca22189909 P 31f3c6c99bab401fb6eb011dea8fb13b: No bootstrap required, opened a new log
I20260809 15:35:44.696995 5963 ts_tablet_manager.cc:1397] T 544cf040ec1d46d8be4c46ca22189909 P 31f3c6c99bab401fb6eb011dea8fb13b: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260809 15:35:44.697212 5965 tablet_bootstrap.cc:492] T 544cf040ec1d46d8be4c46ca22189909 P 66c38532da714eceb1ef3488f4f5263d: No bootstrap required, opened a new log
I20260809 15:35:44.697304 5965 ts_tablet_manager.cc:1397] T 544cf040ec1d46d8be4c46ca22189909 P 66c38532da714eceb1ef3488f4f5263d: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260809 15:35:44.697330 5964 raft_consensus.cc:348] T 544cf040ec1d46d8be4c46ca22189909 P 8ed96760f4584a5ca96e33c6ce9a8f2e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "66c38532da714eceb1ef3488f4f5263d" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 38695 } } peers { permanent_uuid: "31f3c6c99bab401fb6eb011dea8fb13b" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 37671 } } peers { permanent_uuid: "8ed96760f4584a5ca96e33c6ce9a8f2e" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41519 } }
I20260809 15:35:44.697476 5964 raft_consensus.cc:374] T 544cf040ec1d46d8be4c46ca22189909 P 8ed96760f4584a5ca96e33c6ce9a8f2e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 15:35:44.697515 5964 raft_consensus.cc:729] T 544cf040ec1d46d8be4c46ca22189909 P 8ed96760f4584a5ca96e33c6ce9a8f2e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8ed96760f4584a5ca96e33c6ce9a8f2e, State: Initialized, Role: FOLLOWER
I20260809 15:35:44.697485 5963 raft_consensus.cc:348] T 544cf040ec1d46d8be4c46ca22189909 P 31f3c6c99bab401fb6eb011dea8fb13b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "66c38532da714eceb1ef3488f4f5263d" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 38695 } } peers { permanent_uuid: "31f3c6c99bab401fb6eb011dea8fb13b" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 37671 } } peers { permanent_uuid: "8ed96760f4584a5ca96e33c6ce9a8f2e" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41519 } }
I20260809 15:35:44.697602 5963 raft_consensus.cc:374] T 544cf040ec1d46d8be4c46ca22189909 P 31f3c6c99bab401fb6eb011dea8fb13b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 15:35:44.697649 5963 raft_consensus.cc:729] T 544cf040ec1d46d8be4c46ca22189909 P 31f3c6c99bab401fb6eb011dea8fb13b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 31f3c6c99bab401fb6eb011dea8fb13b, State: Initialized, Role: FOLLOWER
I20260809 15:35:44.697677 5964 consensus_queue.cc:260] T 544cf040ec1d46d8be4c46ca22189909 P 8ed96760f4584a5ca96e33c6ce9a8f2e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "66c38532da714eceb1ef3488f4f5263d" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 38695 } } peers { permanent_uuid: "31f3c6c99bab401fb6eb011dea8fb13b" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 37671 } } peers { permanent_uuid: "8ed96760f4584a5ca96e33c6ce9a8f2e" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41519 } }
I20260809 15:35:44.697731 5965 raft_consensus.cc:348] T 544cf040ec1d46d8be4c46ca22189909 P 66c38532da714eceb1ef3488f4f5263d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "66c38532da714eceb1ef3488f4f5263d" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 38695 } } peers { permanent_uuid: "31f3c6c99bab401fb6eb011dea8fb13b" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 37671 } } peers { permanent_uuid: "8ed96760f4584a5ca96e33c6ce9a8f2e" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41519 } }
I20260809 15:35:44.697788 5963 consensus_queue.cc:260] T 544cf040ec1d46d8be4c46ca22189909 P 31f3c6c99bab401fb6eb011dea8fb13b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "66c38532da714eceb1ef3488f4f5263d" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 38695 } } peers { permanent_uuid: "31f3c6c99bab401fb6eb011dea8fb13b" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 37671 } } peers { permanent_uuid: "8ed96760f4584a5ca96e33c6ce9a8f2e" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41519 } }
I20260809 15:35:44.697845 5965 raft_consensus.cc:374] T 544cf040ec1d46d8be4c46ca22189909 P 66c38532da714eceb1ef3488f4f5263d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 15:35:44.697899 5965 raft_consensus.cc:729] T 544cf040ec1d46d8be4c46ca22189909 P 66c38532da714eceb1ef3488f4f5263d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 66c38532da714eceb1ef3488f4f5263d, State: Initialized, Role: FOLLOWER
I20260809 15:35:44.698010 5964 ts_tablet_manager.cc:1428] T 544cf040ec1d46d8be4c46ca22189909 P 8ed96760f4584a5ca96e33c6ce9a8f2e: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260809 15:35:44.698074 5965 consensus_queue.cc:260] T 544cf040ec1d46d8be4c46ca22189909 P 66c38532da714eceb1ef3488f4f5263d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "66c38532da714eceb1ef3488f4f5263d" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 38695 } } peers { permanent_uuid: "31f3c6c99bab401fb6eb011dea8fb13b" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 37671 } } peers { permanent_uuid: "8ed96760f4584a5ca96e33c6ce9a8f2e" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41519 } }
I20260809 15:35:44.698236 5963 ts_tablet_manager.cc:1428] T 544cf040ec1d46d8be4c46ca22189909 P 31f3c6c99bab401fb6eb011dea8fb13b: Time spent starting tablet: real 0.001s user 0.002s sys 0.000s
I20260809 15:35:44.698396 5965 ts_tablet_manager.cc:1428] T 544cf040ec1d46d8be4c46ca22189909 P 66c38532da714eceb1ef3488f4f5263d: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260809 15:35:44.883872 5880 raft_consensus.cc:482] T 544cf040ec1d46d8be4c46ca22189909 P 66c38532da714eceb1ef3488f4f5263d [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 15:35:44.884045 5880 raft_consensus.cc:504] T 544cf040ec1d46d8be4c46ca22189909 P 66c38532da714eceb1ef3488f4f5263d [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "66c38532da714eceb1ef3488f4f5263d" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 38695 } } peers { permanent_uuid: "31f3c6c99bab401fb6eb011dea8fb13b" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 37671 } } peers { permanent_uuid: "8ed96760f4584a5ca96e33c6ce9a8f2e" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41519 } }
I20260809 15:35:44.884461 5880 leader_election.cc:288] T 544cf040ec1d46d8be4c46ca22189909 P 66c38532da714eceb1ef3488f4f5263d [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 31f3c6c99bab401fb6eb011dea8fb13b (127.3.89.194:37671), 8ed96760f4584a5ca96e33c6ce9a8f2e (127.3.89.195:41519)
I20260809 15:35:44.884991 5795 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "544cf040ec1d46d8be4c46ca22189909" candidate_uuid: "66c38532da714eceb1ef3488f4f5263d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8ed96760f4584a5ca96e33c6ce9a8f2e" is_pre_election: true
I20260809 15:35:44.884995 5637 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "544cf040ec1d46d8be4c46ca22189909" candidate_uuid: "66c38532da714eceb1ef3488f4f5263d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "31f3c6c99bab401fb6eb011dea8fb13b" is_pre_election: true
I20260809 15:35:44.885128 5795 raft_consensus.cc:2454] T 544cf040ec1d46d8be4c46ca22189909 P 8ed96760f4584a5ca96e33c6ce9a8f2e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 66c38532da714eceb1ef3488f4f5263d in term 0.
I20260809 15:35:44.885174 5637 raft_consensus.cc:2454] T 544cf040ec1d46d8be4c46ca22189909 P 31f3c6c99bab401fb6eb011dea8fb13b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 66c38532da714eceb1ef3488f4f5263d in term 0.
I20260809 15:35:44.885514 5413 leader_election.cc:302] T 544cf040ec1d46d8be4c46ca22189909 P 66c38532da714eceb1ef3488f4f5263d [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 66c38532da714eceb1ef3488f4f5263d, 8ed96760f4584a5ca96e33c6ce9a8f2e; no voters:
I20260809 15:35:44.885727 5880 raft_consensus.cc:2781] T 544cf040ec1d46d8be4c46ca22189909 P 66c38532da714eceb1ef3488f4f5263d [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260809 15:35:44.885792 5880 raft_consensus.cc:482] T 544cf040ec1d46d8be4c46ca22189909 P 66c38532da714eceb1ef3488f4f5263d [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 15:35:44.885819 5880 raft_consensus.cc:3037] T 544cf040ec1d46d8be4c46ca22189909 P 66c38532da714eceb1ef3488f4f5263d [term 0 FOLLOWER]: Advancing to term 1
I20260809 15:35:44.886883 5880 raft_consensus.cc:504] T 544cf040ec1d46d8be4c46ca22189909 P 66c38532da714eceb1ef3488f4f5263d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "66c38532da714eceb1ef3488f4f5263d" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 38695 } } peers { permanent_uuid: "31f3c6c99bab401fb6eb011dea8fb13b" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 37671 } } peers { permanent_uuid: "8ed96760f4584a5ca96e33c6ce9a8f2e" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41519 } }
I20260809 15:35:44.887251 5880 leader_election.cc:288] T 544cf040ec1d46d8be4c46ca22189909 P 66c38532da714eceb1ef3488f4f5263d [CANDIDATE]: Term 1 election: Requested vote from peers 31f3c6c99bab401fb6eb011dea8fb13b (127.3.89.194:37671), 8ed96760f4584a5ca96e33c6ce9a8f2e (127.3.89.195:41519)
I20260809 15:35:44.887652 5637 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "544cf040ec1d46d8be4c46ca22189909" candidate_uuid: "66c38532da714eceb1ef3488f4f5263d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "31f3c6c99bab401fb6eb011dea8fb13b"
I20260809 15:35:44.887686 5795 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "544cf040ec1d46d8be4c46ca22189909" candidate_uuid: "66c38532da714eceb1ef3488f4f5263d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8ed96760f4584a5ca96e33c6ce9a8f2e"
I20260809 15:35:44.887806 5795 raft_consensus.cc:3037] T 544cf040ec1d46d8be4c46ca22189909 P 8ed96760f4584a5ca96e33c6ce9a8f2e [term 0 FOLLOWER]: Advancing to term 1
I20260809 15:35:44.887804 5637 raft_consensus.cc:3037] T 544cf040ec1d46d8be4c46ca22189909 P 31f3c6c99bab401fb6eb011dea8fb13b [term 0 FOLLOWER]: Advancing to term 1
I20260809 15:35:44.888720 5637 raft_consensus.cc:2454] T 544cf040ec1d46d8be4c46ca22189909 P 31f3c6c99bab401fb6eb011dea8fb13b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 66c38532da714eceb1ef3488f4f5263d in term 1.
I20260809 15:35:44.888721 5795 raft_consensus.cc:2454] T 544cf040ec1d46d8be4c46ca22189909 P 8ed96760f4584a5ca96e33c6ce9a8f2e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 66c38532da714eceb1ef3488f4f5263d in term 1.
I20260809 15:35:44.889031 5413 leader_election.cc:302] T 544cf040ec1d46d8be4c46ca22189909 P 66c38532da714eceb1ef3488f4f5263d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 66c38532da714eceb1ef3488f4f5263d, 8ed96760f4584a5ca96e33c6ce9a8f2e; no voters:
I20260809 15:35:44.889235 5880 raft_consensus.cc:2781] T 544cf040ec1d46d8be4c46ca22189909 P 66c38532da714eceb1ef3488f4f5263d [term 1 FOLLOWER]: Leader election won for term 1
I20260809 15:35:44.889356 5880 raft_consensus.cc:686] T 544cf040ec1d46d8be4c46ca22189909 P 66c38532da714eceb1ef3488f4f5263d [term 1 LEADER]: Becoming Leader. State: Replica: 66c38532da714eceb1ef3488f4f5263d, State: Running, Role: LEADER
I20260809 15:35:44.889510 5880 consensus_queue.cc:237] T 544cf040ec1d46d8be4c46ca22189909 P 66c38532da714eceb1ef3488f4f5263d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "66c38532da714eceb1ef3488f4f5263d" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 38695 } } peers { permanent_uuid: "31f3c6c99bab401fb6eb011dea8fb13b" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 37671 } } peers { permanent_uuid: "8ed96760f4584a5ca96e33c6ce9a8f2e" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41519 } }
I20260809 15:35:44.891324 5317 catalog_manager.cc:5495] T 544cf040ec1d46d8be4c46ca22189909 P 66c38532da714eceb1ef3488f4f5263d reported cstate change: term changed from 0 to 1, leader changed from <none> to 66c38532da714eceb1ef3488f4f5263d (127.3.89.193). New cstate: current_term: 1 leader_uuid: "66c38532da714eceb1ef3488f4f5263d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "66c38532da714eceb1ef3488f4f5263d" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 38695 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "31f3c6c99bab401fb6eb011dea8fb13b" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 37671 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8ed96760f4584a5ca96e33c6ce9a8f2e" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41519 } health_report { overall_health: UNKNOWN } } }
I20260809 15:35:44.936666 5972 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:35:44.933089 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54930 (local address 127.3.89.193:38695)
0809 15:35:44.933414 (+ 325us) server_negotiation.cc:206] Beginning negotiation
0809 15:35:44.933424 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0809 15:35:44.933436 (+ 12us) server_negotiation.cc:407] Connection header received
0809 15:35:44.933481 (+ 45us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 15:35:44.933484 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 15:35:44.933521 (+ 37us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 15:35:44.933613 (+ 92us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 15:35:44.933896 (+ 283us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:35:44.934436 (+ 540us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:35:44.935415 (+ 979us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:35:44.935744 (+ 329us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:35:44.935859 (+ 115us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:35:44.936176 (+ 317us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 15:35:44.936258 (+ 82us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 15:35:44.936344 (+ 86us) server_negotiation.cc:1035] Waiting for connection context
0809 15:35:44.936439 (+ 95us) server_negotiation.cc:299] Negotiation successful
0809 15:35:44.936491 (+ 52us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":240,"thread_start_us":130,"threads_started":1}
I20260809 15:35:44.942744 5795 raft_consensus.cc:1261] T 544cf040ec1d46d8be4c46ca22189909 P 8ed96760f4584a5ca96e33c6ce9a8f2e [term 1 FOLLOWER]: Refusing update from remote peer 66c38532da714eceb1ef3488f4f5263d: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 15:35:44.942755 5637 raft_consensus.cc:1261] T 544cf040ec1d46d8be4c46ca22189909 P 31f3c6c99bab401fb6eb011dea8fb13b [term 1 FOLLOWER]: Refusing update from remote peer 66c38532da714eceb1ef3488f4f5263d: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 15:35:44.943164 5946 consensus_queue.cc:1035] T 544cf040ec1d46d8be4c46ca22189909 P 66c38532da714eceb1ef3488f4f5263d [LEADER]: Connected to new peer: Peer: permanent_uuid: "8ed96760f4584a5ca96e33c6ce9a8f2e" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41519 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 15:35:44.943354 5880 consensus_queue.cc:1035] T 544cf040ec1d46d8be4c46ca22189909 P 66c38532da714eceb1ef3488f4f5263d [LEADER]: Connected to new peer: Peer: permanent_uuid: "31f3c6c99bab401fb6eb011dea8fb13b" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 37671 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 15:35:44.974706 5316 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:45020:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 15:35:44.974934 5316 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:45020:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 15:35:44.976802 5316 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 726056f89ec1474594dd4b44dcfadbfe: Sending DeleteTablet for 3 replicas of tablet 544cf040ec1d46d8be4c46ca22189909
I20260809 15:35:44.977339 5773 tablet_service.cc:1514] Processing DeleteTablet for tablet 544cf040ec1d46d8be4c46ca22189909 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 15:35:44 UTC) from {username='kudu', principal='kudu/127.3.89.254@KRBTEST.COM'} at 127.0.0.1:46744
I20260809 15:35:44.977344 5465 tablet_service.cc:1514] Processing DeleteTablet for tablet 544cf040ec1d46d8be4c46ca22189909 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 15:35:44 UTC) from {username='kudu', principal='kudu/127.3.89.254@KRBTEST.COM'} at 127.0.0.1:54908
I20260809 15:35:44.977537 5953 tablet_replica.cc:330] T 544cf040ec1d46d8be4c46ca22189909 P 8ed96760f4584a5ca96e33c6ce9a8f2e: stopping tablet replica
I20260809 15:35:44.977679 5953 raft_consensus.cc:2229] T 544cf040ec1d46d8be4c46ca22189909 P 8ed96760f4584a5ca96e33c6ce9a8f2e [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 15:35:44.977803 5953 raft_consensus.cc:2258] T 544cf040ec1d46d8be4c46ca22189909 P 8ed96760f4584a5ca96e33c6ce9a8f2e [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 15:35:44.977909 5316 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:45020:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260809 15:35:44.978279 5615 tablet_service.cc:1514] Processing DeleteTablet for tablet 544cf040ec1d46d8be4c46ca22189909 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 15:35:44 UTC) from {username='kudu', principal='kudu/127.3.89.254@KRBTEST.COM'} at 127.0.0.1:42122
W20260809 15:35:44.978319 5316 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260809 15:35:44.978489 5952 tablet_replica.cc:330] T 544cf040ec1d46d8be4c46ca22189909 P 66c38532da714eceb1ef3488f4f5263d: stopping tablet replica
I20260809 15:35:44.978607 5952 raft_consensus.cc:2229] T 544cf040ec1d46d8be4c46ca22189909 P 66c38532da714eceb1ef3488f4f5263d [term 1 LEADER]: Raft consensus shutting down.
I20260809 15:35:44.978621 5953 ts_tablet_manager.cc:1905] T 544cf040ec1d46d8be4c46ca22189909 P 8ed96760f4584a5ca96e33c6ce9a8f2e: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 15:35:44.978797 5952 raft_consensus.cc:2258] T 544cf040ec1d46d8be4c46ca22189909 P 66c38532da714eceb1ef3488f4f5263d [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 15:35:44.979020 5955 tablet_replica.cc:330] T 544cf040ec1d46d8be4c46ca22189909 P 31f3c6c99bab401fb6eb011dea8fb13b: stopping tablet replica
I20260809 15:35:44.979137 5955 raft_consensus.cc:2229] T 544cf040ec1d46d8be4c46ca22189909 P 31f3c6c99bab401fb6eb011dea8fb13b [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 15:35:44.979242 5955 raft_consensus.cc:2258] T 544cf040ec1d46d8be4c46ca22189909 P 31f3c6c99bab401fb6eb011dea8fb13b [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 15:35:44.979869 5952 ts_tablet_manager.cc:1905] T 544cf040ec1d46d8be4c46ca22189909 P 66c38532da714eceb1ef3488f4f5263d: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 15:35:44.980026 5955 ts_tablet_manager.cc:1905] T 544cf040ec1d46d8be4c46ca22189909 P 31f3c6c99bab401fb6eb011dea8fb13b: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 15:35:44.981349 5953 ts_tablet_manager.cc:1918] T 544cf040ec1d46d8be4c46ca22189909 P 8ed96760f4584a5ca96e33c6ce9a8f2e: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260809 15:35:44.981419 5953 log.cc:1196] T 544cf040ec1d46d8be4c46ca22189909 P 8ed96760f4584a5ca96e33c6ce9a8f2e: Deleting WAL directory at /tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786289732823074-3431-0/minicluster-data/ts-2/wal/wals/544cf040ec1d46d8be4c46ca22189909
I20260809 15:35:44.981709 5953 ts_tablet_manager.cc:1939] T 544cf040ec1d46d8be4c46ca22189909 P 8ed96760f4584a5ca96e33c6ce9a8f2e: Deleting consensus metadata
I20260809 15:35:44.982498 5955 ts_tablet_manager.cc:1918] T 544cf040ec1d46d8be4c46ca22189909 P 31f3c6c99bab401fb6eb011dea8fb13b: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260809 15:35:44.982582 5955 log.cc:1196] T 544cf040ec1d46d8be4c46ca22189909 P 31f3c6c99bab401fb6eb011dea8fb13b: Deleting WAL directory at /tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786289732823074-3431-0/minicluster-data/ts-1/wal/wals/544cf040ec1d46d8be4c46ca22189909
I20260809 15:35:44.982642 5952 ts_tablet_manager.cc:1918] T 544cf040ec1d46d8be4c46ca22189909 P 66c38532da714eceb1ef3488f4f5263d: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260809 15:35:44.982715 5952 log.cc:1196] T 544cf040ec1d46d8be4c46ca22189909 P 66c38532da714eceb1ef3488f4f5263d: Deleting WAL directory at /tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786289732823074-3431-0/minicluster-data/ts-0/wal/wals/544cf040ec1d46d8be4c46ca22189909
I20260809 15:35:44.982846 5955 ts_tablet_manager.cc:1939] T 544cf040ec1d46d8be4c46ca22189909 P 31f3c6c99bab401fb6eb011dea8fb13b: Deleting consensus metadata
I20260809 15:35:44.983161 5465 tablet_service.cc:1467] Processing CreateTablet for tablet a6c0e2eebec5409283774dda173132d2 (DEFAULT_TABLE table=test-table [id=c13751c267da49dfa46782d2a16e2da9]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 15:35:44.983330 5305 catalog_manager.cc:4841] TS 8ed96760f4584a5ca96e33c6ce9a8f2e (127.3.89.195:41519): tablet 544cf040ec1d46d8be4c46ca22189909 (table test-table [id=6a7cb370114e408badacc86ec7eb2ae7]) successfully deleted
I20260809 15:35:44.983438 5465 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a6c0e2eebec5409283774dda173132d2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 15:35:44.983422 5615 tablet_service.cc:1467] Processing CreateTablet for tablet a6c0e2eebec5409283774dda173132d2 (DEFAULT_TABLE table=test-table [id=c13751c267da49dfa46782d2a16e2da9]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 15:35:44.983628 5615 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a6c0e2eebec5409283774dda173132d2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 15:35:44.983585 5773 tablet_service.cc:1467] Processing CreateTablet for tablet a6c0e2eebec5409283774dda173132d2 (DEFAULT_TABLE table=test-table [id=c13751c267da49dfa46782d2a16e2da9]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 15:35:44.983830 5773 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a6c0e2eebec5409283774dda173132d2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 15:35:44.985155 5963 tablet_bootstrap.cc:492] T a6c0e2eebec5409283774dda173132d2 P 31f3c6c99bab401fb6eb011dea8fb13b: Bootstrap starting.
I20260809 15:35:44.985610 5965 tablet_bootstrap.cc:492] T a6c0e2eebec5409283774dda173132d2 P 66c38532da714eceb1ef3488f4f5263d: Bootstrap starting.
I20260809 15:35:44.986112 5963 tablet_bootstrap.cc:654] T a6c0e2eebec5409283774dda173132d2 P 31f3c6c99bab401fb6eb011dea8fb13b: Neither blocks nor log segments found. Creating new log.
I20260809 15:35:44.986384 5952 ts_tablet_manager.cc:1939] T 544cf040ec1d46d8be4c46ca22189909 P 66c38532da714eceb1ef3488f4f5263d: Deleting consensus metadata
I20260809 15:35:44.986479 5965 tablet_bootstrap.cc:654] T a6c0e2eebec5409283774dda173132d2 P 66c38532da714eceb1ef3488f4f5263d: Neither blocks nor log segments found. Creating new log.
I20260809 15:35:44.987054 5964 tablet_bootstrap.cc:492] T a6c0e2eebec5409283774dda173132d2 P 8ed96760f4584a5ca96e33c6ce9a8f2e: Bootstrap starting.
I20260809 15:35:44.987421 5305 catalog_manager.cc:4841] TS 31f3c6c99bab401fb6eb011dea8fb13b (127.3.89.194:37671): tablet 544cf040ec1d46d8be4c46ca22189909 (table test-table [id=6a7cb370114e408badacc86ec7eb2ae7]) successfully deleted
I20260809 15:35:44.987493 5303 catalog_manager.cc:4841] TS 66c38532da714eceb1ef3488f4f5263d (127.3.89.193:38695): tablet 544cf040ec1d46d8be4c46ca22189909 (table test-table [id=6a7cb370114e408badacc86ec7eb2ae7]) successfully deleted
I20260809 15:35:44.987455 5963 tablet_bootstrap.cc:492] T a6c0e2eebec5409283774dda173132d2 P 31f3c6c99bab401fb6eb011dea8fb13b: No bootstrap required, opened a new log
I20260809 15:35:44.987628 5963 ts_tablet_manager.cc:1397] T a6c0e2eebec5409283774dda173132d2 P 31f3c6c99bab401fb6eb011dea8fb13b: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260809 15:35:44.987970 5964 tablet_bootstrap.cc:654] T a6c0e2eebec5409283774dda173132d2 P 8ed96760f4584a5ca96e33c6ce9a8f2e: Neither blocks nor log segments found. Creating new log.
I20260809 15:35:44.988104 5963 raft_consensus.cc:348] T a6c0e2eebec5409283774dda173132d2 P 31f3c6c99bab401fb6eb011dea8fb13b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "66c38532da714eceb1ef3488f4f5263d" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 38695 } } peers { permanent_uuid: "31f3c6c99bab401fb6eb011dea8fb13b" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 37671 } } peers { permanent_uuid: "8ed96760f4584a5ca96e33c6ce9a8f2e" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41519 } }
I20260809 15:35:44.988225 5963 raft_consensus.cc:374] T a6c0e2eebec5409283774dda173132d2 P 31f3c6c99bab401fb6eb011dea8fb13b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 15:35:44.988287 5963 raft_consensus.cc:729] T a6c0e2eebec5409283774dda173132d2 P 31f3c6c99bab401fb6eb011dea8fb13b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 31f3c6c99bab401fb6eb011dea8fb13b, State: Initialized, Role: FOLLOWER
I20260809 15:35:44.988418 5963 consensus_queue.cc:260] T a6c0e2eebec5409283774dda173132d2 P 31f3c6c99bab401fb6eb011dea8fb13b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "66c38532da714eceb1ef3488f4f5263d" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 38695 } } peers { permanent_uuid: "31f3c6c99bab401fb6eb011dea8fb13b" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 37671 } } peers { permanent_uuid: "8ed96760f4584a5ca96e33c6ce9a8f2e" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41519 } }
I20260809 15:35:44.988713 5963 ts_tablet_manager.cc:1428] T a6c0e2eebec5409283774dda173132d2 P 31f3c6c99bab401fb6eb011dea8fb13b: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20260809 15:35:44.989203 5964 tablet_bootstrap.cc:492] T a6c0e2eebec5409283774dda173132d2 P 8ed96760f4584a5ca96e33c6ce9a8f2e: No bootstrap required, opened a new log
I20260809 15:35:44.989333 5964 ts_tablet_manager.cc:1397] T a6c0e2eebec5409283774dda173132d2 P 8ed96760f4584a5ca96e33c6ce9a8f2e: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260809 15:35:44.989665 5965 tablet_bootstrap.cc:492] T a6c0e2eebec5409283774dda173132d2 P 66c38532da714eceb1ef3488f4f5263d: No bootstrap required, opened a new log
I20260809 15:35:44.989688 5964 raft_consensus.cc:348] T a6c0e2eebec5409283774dda173132d2 P 8ed96760f4584a5ca96e33c6ce9a8f2e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "66c38532da714eceb1ef3488f4f5263d" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 38695 } } peers { permanent_uuid: "31f3c6c99bab401fb6eb011dea8fb13b" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 37671 } } peers { permanent_uuid: "8ed96760f4584a5ca96e33c6ce9a8f2e" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41519 } }
I20260809 15:35:44.989761 5965 ts_tablet_manager.cc:1397] T a6c0e2eebec5409283774dda173132d2 P 66c38532da714eceb1ef3488f4f5263d: Time spent bootstrapping tablet: real 0.004s user 0.001s sys 0.000s
I20260809 15:35:44.989817 5964 raft_consensus.cc:374] T a6c0e2eebec5409283774dda173132d2 P 8ed96760f4584a5ca96e33c6ce9a8f2e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 15:35:44.989864 5964 raft_consensus.cc:729] T a6c0e2eebec5409283774dda173132d2 P 8ed96760f4584a5ca96e33c6ce9a8f2e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8ed96760f4584a5ca96e33c6ce9a8f2e, State: Initialized, Role: FOLLOWER
I20260809 15:35:44.990002 5964 consensus_queue.cc:260] T a6c0e2eebec5409283774dda173132d2 P 8ed96760f4584a5ca96e33c6ce9a8f2e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "66c38532da714eceb1ef3488f4f5263d" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 38695 } } peers { permanent_uuid: "31f3c6c99bab401fb6eb011dea8fb13b" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 37671 } } peers { permanent_uuid: "8ed96760f4584a5ca96e33c6ce9a8f2e" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41519 } }
I20260809 15:35:44.990264 5964 ts_tablet_manager.cc:1428] T a6c0e2eebec5409283774dda173132d2 P 8ed96760f4584a5ca96e33c6ce9a8f2e: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260809 15:35:44.990221 5965 raft_consensus.cc:348] T a6c0e2eebec5409283774dda173132d2 P 66c38532da714eceb1ef3488f4f5263d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "66c38532da714eceb1ef3488f4f5263d" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 38695 } } peers { permanent_uuid: "31f3c6c99bab401fb6eb011dea8fb13b" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 37671 } } peers { permanent_uuid: "8ed96760f4584a5ca96e33c6ce9a8f2e" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41519 } }
I20260809 15:35:44.990327 5965 raft_consensus.cc:374] T a6c0e2eebec5409283774dda173132d2 P 66c38532da714eceb1ef3488f4f5263d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 15:35:44.990373 5965 raft_consensus.cc:729] T a6c0e2eebec5409283774dda173132d2 P 66c38532da714eceb1ef3488f4f5263d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 66c38532da714eceb1ef3488f4f5263d, State: Initialized, Role: FOLLOWER
I20260809 15:35:44.990494 5965 consensus_queue.cc:260] T a6c0e2eebec5409283774dda173132d2 P 66c38532da714eceb1ef3488f4f5263d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "66c38532da714eceb1ef3488f4f5263d" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 38695 } } peers { permanent_uuid: "31f3c6c99bab401fb6eb011dea8fb13b" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 37671 } } peers { permanent_uuid: "8ed96760f4584a5ca96e33c6ce9a8f2e" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41519 } }
I20260809 15:35:44.990794 5965 ts_tablet_manager.cc:1428] T a6c0e2eebec5409283774dda173132d2 P 66c38532da714eceb1ef3488f4f5263d: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260809 15:35:45.049557 5943 raft_consensus.cc:482] T a6c0e2eebec5409283774dda173132d2 P 66c38532da714eceb1ef3488f4f5263d [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 15:35:45.049749 5943 raft_consensus.cc:504] T a6c0e2eebec5409283774dda173132d2 P 66c38532da714eceb1ef3488f4f5263d [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "66c38532da714eceb1ef3488f4f5263d" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 38695 } } peers { permanent_uuid: "31f3c6c99bab401fb6eb011dea8fb13b" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 37671 } } peers { permanent_uuid: "8ed96760f4584a5ca96e33c6ce9a8f2e" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41519 } }
I20260809 15:35:45.050235 5943 leader_election.cc:288] T a6c0e2eebec5409283774dda173132d2 P 66c38532da714eceb1ef3488f4f5263d [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 31f3c6c99bab401fb6eb011dea8fb13b (127.3.89.194:37671), 8ed96760f4584a5ca96e33c6ce9a8f2e (127.3.89.195:41519)
I20260809 15:35:45.050778 5637 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a6c0e2eebec5409283774dda173132d2" candidate_uuid: "66c38532da714eceb1ef3488f4f5263d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "31f3c6c99bab401fb6eb011dea8fb13b" is_pre_election: true
I20260809 15:35:45.050952 5637 raft_consensus.cc:2454] T a6c0e2eebec5409283774dda173132d2 P 31f3c6c99bab401fb6eb011dea8fb13b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 66c38532da714eceb1ef3488f4f5263d in term 0.
I20260809 15:35:45.051146 5795 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a6c0e2eebec5409283774dda173132d2" candidate_uuid: "66c38532da714eceb1ef3488f4f5263d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8ed96760f4584a5ca96e33c6ce9a8f2e" is_pre_election: true
I20260809 15:35:45.051262 5413 leader_election.cc:302] T a6c0e2eebec5409283774dda173132d2 P 66c38532da714eceb1ef3488f4f5263d [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 31f3c6c99bab401fb6eb011dea8fb13b, 66c38532da714eceb1ef3488f4f5263d; no voters:
I20260809 15:35:45.051316 5795 raft_consensus.cc:2454] T a6c0e2eebec5409283774dda173132d2 P 8ed96760f4584a5ca96e33c6ce9a8f2e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 66c38532da714eceb1ef3488f4f5263d in term 0.
I20260809 15:35:45.051435 5943 raft_consensus.cc:2781] T a6c0e2eebec5409283774dda173132d2 P 66c38532da714eceb1ef3488f4f5263d [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260809 15:35:45.051501 5943 raft_consensus.cc:482] T a6c0e2eebec5409283774dda173132d2 P 66c38532da714eceb1ef3488f4f5263d [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 15:35:45.051553 5943 raft_consensus.cc:3037] T a6c0e2eebec5409283774dda173132d2 P 66c38532da714eceb1ef3488f4f5263d [term 0 FOLLOWER]: Advancing to term 1
I20260809 15:35:45.052664 5943 raft_consensus.cc:504] T a6c0e2eebec5409283774dda173132d2 P 66c38532da714eceb1ef3488f4f5263d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "66c38532da714eceb1ef3488f4f5263d" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 38695 } } peers { permanent_uuid: "31f3c6c99bab401fb6eb011dea8fb13b" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 37671 } } peers { permanent_uuid: "8ed96760f4584a5ca96e33c6ce9a8f2e" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41519 } }
I20260809 15:35:45.052991 5943 leader_election.cc:288] T a6c0e2eebec5409283774dda173132d2 P 66c38532da714eceb1ef3488f4f5263d [CANDIDATE]: Term 1 election: Requested vote from peers 31f3c6c99bab401fb6eb011dea8fb13b (127.3.89.194:37671), 8ed96760f4584a5ca96e33c6ce9a8f2e (127.3.89.195:41519)
I20260809 15:35:45.053306 5637 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a6c0e2eebec5409283774dda173132d2" candidate_uuid: "66c38532da714eceb1ef3488f4f5263d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "31f3c6c99bab401fb6eb011dea8fb13b"
I20260809 15:35:45.053352 5795 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a6c0e2eebec5409283774dda173132d2" candidate_uuid: "66c38532da714eceb1ef3488f4f5263d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8ed96760f4584a5ca96e33c6ce9a8f2e"
I20260809 15:35:45.053424 5637 raft_consensus.cc:3037] T a6c0e2eebec5409283774dda173132d2 P 31f3c6c99bab401fb6eb011dea8fb13b [term 0 FOLLOWER]: Advancing to term 1
I20260809 15:35:45.053510 5795 raft_consensus.cc:3037] T a6c0e2eebec5409283774dda173132d2 P 8ed96760f4584a5ca96e33c6ce9a8f2e [term 0 FOLLOWER]: Advancing to term 1
I20260809 15:35:45.054351 5637 raft_consensus.cc:2454] T a6c0e2eebec5409283774dda173132d2 P 31f3c6c99bab401fb6eb011dea8fb13b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 66c38532da714eceb1ef3488f4f5263d in term 1.
I20260809 15:35:45.054527 5795 raft_consensus.cc:2454] T a6c0e2eebec5409283774dda173132d2 P 8ed96760f4584a5ca96e33c6ce9a8f2e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 66c38532da714eceb1ef3488f4f5263d in term 1.
I20260809 15:35:45.054648 5413 leader_election.cc:302] T a6c0e2eebec5409283774dda173132d2 P 66c38532da714eceb1ef3488f4f5263d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 31f3c6c99bab401fb6eb011dea8fb13b, 66c38532da714eceb1ef3488f4f5263d; no voters:
I20260809 15:35:45.054829 5943 raft_consensus.cc:2781] T a6c0e2eebec5409283774dda173132d2 P 66c38532da714eceb1ef3488f4f5263d [term 1 FOLLOWER]: Leader election won for term 1
I20260809 15:35:45.054982 5943 raft_consensus.cc:686] T a6c0e2eebec5409283774dda173132d2 P 66c38532da714eceb1ef3488f4f5263d [term 1 LEADER]: Becoming Leader. State: Replica: 66c38532da714eceb1ef3488f4f5263d, State: Running, Role: LEADER
I20260809 15:35:45.055159 5943 consensus_queue.cc:237] T a6c0e2eebec5409283774dda173132d2 P 66c38532da714eceb1ef3488f4f5263d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "66c38532da714eceb1ef3488f4f5263d" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 38695 } } peers { permanent_uuid: "31f3c6c99bab401fb6eb011dea8fb13b" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 37671 } } peers { permanent_uuid: "8ed96760f4584a5ca96e33c6ce9a8f2e" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41519 } }
I20260809 15:35:45.056754 5318 catalog_manager.cc:5495] T a6c0e2eebec5409283774dda173132d2 P 66c38532da714eceb1ef3488f4f5263d reported cstate change: term changed from 0 to 1, leader changed from <none> to 66c38532da714eceb1ef3488f4f5263d (127.3.89.193). New cstate: current_term: 1 leader_uuid: "66c38532da714eceb1ef3488f4f5263d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "66c38532da714eceb1ef3488f4f5263d" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 38695 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "31f3c6c99bab401fb6eb011dea8fb13b" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 37671 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8ed96760f4584a5ca96e33c6ce9a8f2e" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41519 } health_report { overall_health: UNKNOWN } } }
I20260809 15:35:45.081647 5795 raft_consensus.cc:1261] T a6c0e2eebec5409283774dda173132d2 P 8ed96760f4584a5ca96e33c6ce9a8f2e [term 1 FOLLOWER]: Refusing update from remote peer 66c38532da714eceb1ef3488f4f5263d: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 15:35:45.081671 5637 raft_consensus.cc:1261] T a6c0e2eebec5409283774dda173132d2 P 31f3c6c99bab401fb6eb011dea8fb13b [term 1 FOLLOWER]: Refusing update from remote peer 66c38532da714eceb1ef3488f4f5263d: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 15:35:45.082118 5880 consensus_queue.cc:1035] T a6c0e2eebec5409283774dda173132d2 P 66c38532da714eceb1ef3488f4f5263d [LEADER]: Connected to new peer: Peer: permanent_uuid: "31f3c6c99bab401fb6eb011dea8fb13b" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 37671 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 15:35:45.082276 5943 consensus_queue.cc:1035] T a6c0e2eebec5409283774dda173132d2 P 66c38532da714eceb1ef3488f4f5263d [LEADER]: Connected to new peer: Peer: permanent_uuid: "8ed96760f4584a5ca96e33c6ce9a8f2e" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41519 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20260809 15:35:45.094501 5935 tablet_replica.cc:1298] Aborted: operation has been aborted: cancelling pending write operations
I20260809 15:35:46.645782 5318 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:45020:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 15:35:46.645979 5318 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:45020:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 15:35:46.648566 5318 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 726056f89ec1474594dd4b44dcfadbfe: Sending DeleteTablet for 3 replicas of tablet a6c0e2eebec5409283774dda173132d2
I20260809 15:35:46.649187 5465 tablet_service.cc:1514] Processing DeleteTablet for tablet a6c0e2eebec5409283774dda173132d2 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 15:35:46 UTC) from {username='kudu', principal='kudu/127.3.89.254@KRBTEST.COM'} at 127.0.0.1:54908
I20260809 15:35:46.649334 5615 tablet_service.cc:1514] Processing DeleteTablet for tablet a6c0e2eebec5409283774dda173132d2 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 15:35:46 UTC) from {username='kudu', principal='kudu/127.3.89.254@KRBTEST.COM'} at 127.0.0.1:42122
I20260809 15:35:46.649621 5773 tablet_service.cc:1514] Processing DeleteTablet for tablet a6c0e2eebec5409283774dda173132d2 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 15:35:46 UTC) from {username='kudu', principal='kudu/127.3.89.254@KRBTEST.COM'} at 127.0.0.1:46744
I20260809 15:35:46.649885 3431 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskm314YR/build/debug/bin/kudu with pid 5393
I20260809 15:35:46.651504 6012 tablet_replica.cc:330] T a6c0e2eebec5409283774dda173132d2 P 8ed96760f4584a5ca96e33c6ce9a8f2e: stopping tablet replica
I20260809 15:35:46.651674 6012 raft_consensus.cc:2229] T a6c0e2eebec5409283774dda173132d2 P 8ed96760f4584a5ca96e33c6ce9a8f2e [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 15:35:46.651796 6012 raft_consensus.cc:2258] T a6c0e2eebec5409283774dda173132d2 P 8ed96760f4584a5ca96e33c6ce9a8f2e [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 15:35:46.652042 6010 tablet_replica.cc:330] T a6c0e2eebec5409283774dda173132d2 P 31f3c6c99bab401fb6eb011dea8fb13b: stopping tablet replica
I20260809 15:35:46.652195 6010 raft_consensus.cc:2229] T a6c0e2eebec5409283774dda173132d2 P 31f3c6c99bab401fb6eb011dea8fb13b [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 15:35:46.652330 6010 raft_consensus.cc:2258] T a6c0e2eebec5409283774dda173132d2 P 31f3c6c99bab401fb6eb011dea8fb13b [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 15:35:46.652882 6012 ts_tablet_manager.cc:1905] T a6c0e2eebec5409283774dda173132d2 P 8ed96760f4584a5ca96e33c6ce9a8f2e: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 15:35:46.655321 6012 ts_tablet_manager.cc:1918] T a6c0e2eebec5409283774dda173132d2 P 8ed96760f4584a5ca96e33c6ce9a8f2e: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260809 15:35:46.655352 6010 ts_tablet_manager.cc:1905] T a6c0e2eebec5409283774dda173132d2 P 31f3c6c99bab401fb6eb011dea8fb13b: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 15:35:46.655417 6012 log.cc:1196] T a6c0e2eebec5409283774dda173132d2 P 8ed96760f4584a5ca96e33c6ce9a8f2e: Deleting WAL directory at /tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786289732823074-3431-0/minicluster-data/ts-2/wal/wals/a6c0e2eebec5409283774dda173132d2
I20260809 15:35:46.655718 6012 ts_tablet_manager.cc:1939] T a6c0e2eebec5409283774dda173132d2 P 8ed96760f4584a5ca96e33c6ce9a8f2e: Deleting consensus metadata
I20260809 15:35:46.656595 5305 catalog_manager.cc:4841] TS 8ed96760f4584a5ca96e33c6ce9a8f2e (127.3.89.195:41519): tablet a6c0e2eebec5409283774dda173132d2 (table test-table [id=c13751c267da49dfa46782d2a16e2da9]) successfully deleted
I20260809 15:35:46.657509 6010 ts_tablet_manager.cc:1918] T a6c0e2eebec5409283774dda173132d2 P 31f3c6c99bab401fb6eb011dea8fb13b: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260809 15:35:46.657601 6010 log.cc:1196] T a6c0e2eebec5409283774dda173132d2 P 31f3c6c99bab401fb6eb011dea8fb13b: Deleting WAL directory at /tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786289732823074-3431-0/minicluster-data/ts-1/wal/wals/a6c0e2eebec5409283774dda173132d2
I20260809 15:35:46.657855 6010 ts_tablet_manager.cc:1939] T a6c0e2eebec5409283774dda173132d2 P 31f3c6c99bab401fb6eb011dea8fb13b: Deleting consensus metadata
I20260809 15:35:46.658493 5305 catalog_manager.cc:4841] TS 31f3c6c99bab401fb6eb011dea8fb13b (127.3.89.194:37671): tablet a6c0e2eebec5409283774dda173132d2 (table test-table [id=c13751c267da49dfa46782d2a16e2da9]) successfully deleted
W20260809 15:35:46.660933 5303 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.3.89.193:38695 (error 108)
I20260809 15:35:46.661226 3431 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskm314YR/build/debug/bin/kudu with pid 5546
I20260809 15:35:46.664086 6014 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:35:46.663639 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.89.193:38695 (local address 127.0.0.1:54932)
0809 15:35:46.663902 (+ 263us) negotiation.cc:107] Waiting for socket to connect
0809 15:35:46.663998 (+ 96us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.89.193:38695: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":168,"thread_start_us":101,"threads_started":1}
W20260809 15:35:46.664239 5303 catalog_manager.cc:4584] TS 66c38532da714eceb1ef3488f4f5263d (127.3.89.193:38695): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet a6c0e2eebec5409283774dda173132d2: Network error: Client connection negotiation failed: client connection to 127.3.89.193:38695: connect: Connection refused (error 111)
I20260809 15:35:46.668262 3431 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskm314YR/build/debug/bin/kudu with pid 5698
I20260809 15:35:46.677486 3431 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskm314YR/build/debug/bin/kudu with pid 5284
2026-08-09T15:35:46Z chronyd exiting
[ OK ] SecurityITest.TxnSmokeWithDifferentUserTypes (6222 ms)
[ RUN ] SecurityITest.TestNoKerberosCredentials
Loading random data
Initializing database '/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786289732823074-3431-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 15:35:46 dist-test-slave-2w3w krb5kdc[6017](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 15:35:46 dist-test-slave-2w3w krb5kdc[6017](info): set up 2 sockets
Aug 09 15:35:46 dist-test-slave-2w3w krb5kdc[6017](info): commencing operation
krb5kdc: starting...
W20260809 15:35:47.743973 3431 mini_kdc.cc:121] Time spent starting KDC: real 1.041s user 0.001s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786289732823074-3431-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786289732823074-3431-0/krb5kdc/test-user.keytab.
Aug 09 15:35:47 dist-test-slave-2w3w krb5kdc[6017](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786289747, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T15:35:47Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T15:35:47Z Disabled control of system clock
WARNING: no policy specified for kudu/127.3.89.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.3.89.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.3.89.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786289732823074-3431-0/krb5kdc/kudu_127.3.89.254.keytab.
Entry for principal kudu/127.3.89.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786289732823074-3431-0/krb5kdc/kudu_127.3.89.254.keytab.
I20260809 15:35:47.867859 3431 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskm314YR/build/debug/bin/kudu
/tmp/dist-test-taskm314YR/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786289732823074-3431-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786289732823074-3431-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786289732823074-3431-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786289732823074-3431-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.3.89.254:42289
--webserver_interface=127.3.89.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786289732823074-3431-0/krb5kdc/kudu_127.3.89.254.keytab
--principal=kudu/127.3.89.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.3.89.212:43231
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.3.89.254:42289
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786289732823074-3431-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786289732823074-3431-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786289732823074-3431-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 15:35:47.977722 6031 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 15:35:47.977990 6031 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 15:35:47.978031 6031 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 15:35:47.981945 6031 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 15:35:47.982007 6031 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 15:35:47.982028 6031 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 15:35:47.982044 6031 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 15:35:47.982059 6031 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 15:35:47.987664 6031 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.3.89.212:43231
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786289732823074-3431-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786289732823074-3431-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.3.89.254:42289
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786289732823074-3431-0/krb5kdc/kudu_127.3.89.254.keytab
--principal=kudu/127.3.89.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.3.89.254:42289
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786289732823074-3431-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.3.89.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.6031
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786289732823074-3431-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 09 Aug 2026 14:46:55 UTC on ccycloud.quasar-yswkmf.root.comops.site
build id 4576540
I20260809 15:35:47.988785 6031 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 15:35:47.989845 6031 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 15:35:47.999178 6037 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 15:35:47.999198 6036 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 15:35:47.999178 6031 server_base.cc:1034] running on GCE node
W20260809 15:35:47.999167 6039 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 15:35:48.000057 6031 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 15:35:48.001235 6031 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 15:35:48.002424 6031 hybrid_clock.cc:648] HybridClock initialized: now 1786289748002417 us; error 30 us; skew 500 ppm
Aug 09 15:35:48 dist-test-slave-2w3w krb5kdc[6017](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786289748, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.89.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 15:35:48.010123 6031 init.cc:377] Logged in from keytab as kudu/127.3.89.254@KRBTEST.COM (short username kudu)
I20260809 15:35:48.011080 6031 webserver.cc:460] Webserver started at http://127.3.89.254:39475/ using document root <none> and password file <none>
I20260809 15:35:48.011637 6031 fs_manager.cc:359] Metadata directory not provided
I20260809 15:35:48.011729 6031 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 15:35:48.011963 6031 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 15:35:48.014122 6031 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786289732823074-3431-0/minicluster-data/master-0/data/instance:
uuid: "7443e0f995c14523bb438d06089c5035"
format_stamp: "Formatted at 2026-08-09 15:35:48 on dist-test-slave-2w3w"
server_key: "bc96f1829798dbda0d70cb037f1512aa"
server_key_iv: "7b70c19601b2938d0802d9cd7fff49da"
server_key_version: "encryptionkey@0"
I20260809 15:35:48.014667 6031 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786289732823074-3431-0/minicluster-data/master-0/wal/instance:
uuid: "7443e0f995c14523bb438d06089c5035"
format_stamp: "Formatted at 2026-08-09 15:35:48 on dist-test-slave-2w3w"
server_key: "bc96f1829798dbda0d70cb037f1512aa"
server_key_iv: "7b70c19601b2938d0802d9cd7fff49da"
server_key_version: "encryptionkey@0"
I20260809 15:35:48.018260 6031 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.006s sys 0.000s
I20260809 15:35:48.020617 6047 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 15:35:48.021783 6031 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260809 15:35:48.021916 6031 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786289732823074-3431-0/minicluster-data/master-0/data,/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786289732823074-3431-0/minicluster-data/master-0/wal
uuid: "7443e0f995c14523bb438d06089c5035"
format_stamp: "Formatted at 2026-08-09 15:35:48 on dist-test-slave-2w3w"
server_key: "bc96f1829798dbda0d70cb037f1512aa"
server_key_iv: "7b70c19601b2938d0802d9cd7fff49da"
server_key_version: "encryptionkey@0"
I20260809 15:35:48.022019 6031 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786289732823074-3431-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786289732823074-3431-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786289732823074-3431-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 15:35:48.047984 6031 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 15:35:48.051285 6031 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 15:35:48.051486 6031 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 15:35:48.058547 6031 rpc_server.cc:307] RPC server started. Bound to: 127.3.89.254:42289
I20260809 15:35:48.058578 6104 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.3.89.254:42289 every 8 connection(s)
I20260809 15:35:48.059516 6031 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786289732823074-3431-0/minicluster-data/master-0/data/info.pb
I20260809 15:35:48.062234 6105 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 15:35:48.064388 3431 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskm314YR/build/debug/bin/kudu as pid 6031
I20260809 15:35:48.064513 3431 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786289732823074-3431-0/minicluster-data/master-0/wal/instance
I20260809 15:35:48.064790 3431 external_mini_cluster.cc:1428] Setting key 96bcdba8bdb2f1f0275ae129553f3880
I20260809 15:35:48.068094 6105 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7443e0f995c14523bb438d06089c5035: Bootstrap starting.
I20260809 15:35:48.070514 6105 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 7443e0f995c14523bb438d06089c5035: Neither blocks nor log segments found. Creating new log.
I20260809 15:35:48.071360 6105 log.cc:824] T 00000000000000000000000000000000 P 7443e0f995c14523bb438d06089c5035: Log is configured to *not* fsync() on all Append() calls
I20260809 15:35:48.073585 6105 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7443e0f995c14523bb438d06089c5035: No bootstrap required, opened a new log
Aug 09 15:35:48 dist-test-slave-2w3w krb5kdc[6017](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786289747, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.3.89.254@KRBTEST.COM
I20260809 15:35:48.076628 6105 raft_consensus.cc:348] T 00000000000000000000000000000000 P 7443e0f995c14523bb438d06089c5035 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7443e0f995c14523bb438d06089c5035" member_type: VOTER last_known_addr { host: "127.3.89.254" port: 42289 } }
I20260809 15:35:48.076912 6105 raft_consensus.cc:374] T 00000000000000000000000000000000 P 7443e0f995c14523bb438d06089c5035 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 15:35:48.076994 6105 raft_consensus.cc:729] T 00000000000000000000000000000000 P 7443e0f995c14523bb438d06089c5035 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7443e0f995c14523bb438d06089c5035, State: Initialized, Role: FOLLOWER
I20260809 15:35:48.077447 6105 consensus_queue.cc:260] T 00000000000000000000000000000000 P 7443e0f995c14523bb438d06089c5035 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7443e0f995c14523bb438d06089c5035" member_type: VOTER last_known_addr { host: "127.3.89.254" port: 42289 } }
I20260809 15:35:48.077600 6105 raft_consensus.cc:388] T 00000000000000000000000000000000 P 7443e0f995c14523bb438d06089c5035 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 15:35:48.077700 6105 raft_consensus.cc:482] T 00000000000000000000000000000000 P 7443e0f995c14523bb438d06089c5035 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 15:35:48.077857 6105 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 7443e0f995c14523bb438d06089c5035 [term 0 FOLLOWER]: Advancing to term 1
I20260809 15:35:48.078711 6105 raft_consensus.cc:504] T 00000000000000000000000000000000 P 7443e0f995c14523bb438d06089c5035 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7443e0f995c14523bb438d06089c5035" member_type: VOTER last_known_addr { host: "127.3.89.254" port: 42289 } }
I20260809 15:35:48.079110 6105 leader_election.cc:302] T 00000000000000000000000000000000 P 7443e0f995c14523bb438d06089c5035 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 7443e0f995c14523bb438d06089c5035; no voters:
I20260809 15:35:48.079463 6105 leader_election.cc:288] T 00000000000000000000000000000000 P 7443e0f995c14523bb438d06089c5035 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 15:35:48.079552 6110 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 7443e0f995c14523bb438d06089c5035 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 15:35:48.079757 6110 raft_consensus.cc:686] T 00000000000000000000000000000000 P 7443e0f995c14523bb438d06089c5035 [term 1 LEADER]: Becoming Leader. State: Replica: 7443e0f995c14523bb438d06089c5035, State: Running, Role: LEADER
I20260809 15:35:48.080502 6108 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:35:48.066051 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37732 (local address 127.3.89.254:42289)
0809 15:35:48.066444 (+ 393us) server_negotiation.cc:206] Beginning negotiation
0809 15:35:48.066459 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0809 15:35:48.066494 (+ 35us) server_negotiation.cc:407] Connection header received
0809 15:35:48.067171 (+ 677us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 15:35:48.067188 (+ 17us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 15:35:48.067491 (+ 303us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 15:35:48.067757 (+ 266us) server_negotiation.cc:226] Negotiated authn=SASL
0809 15:35:48.068685 (+ 928us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:35:48.069634 (+ 949us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:35:48.070799 (+ 1165us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:35:48.071294 (+ 495us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:35:48.071409 (+ 115us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:35:48.074422 (+ 3013us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 15:35:48.074426 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 15:35:48.074438 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 15:35:48.074463 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0809 15:35:48.077203 (+ 2740us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 15:35:48.077648 (+ 445us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 15:35:48.077653 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 15:35:48.077658 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0809 15:35:48.077776 (+ 118us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 15:35:48.078115 (+ 339us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 15:35:48.078119 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 15:35:48.078120 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 15:35:48.078501 (+ 381us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 15:35:48.078727 (+ 226us) server_negotiation.cc:1035] Waiting for connection context
0809 15:35:48.079053 (+ 326us) server_negotiation.cc:299] Negotiation successful
0809 15:35:48.079363 (+ 310us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":260,"thread_start_us":123,"threads_started":1}
I20260809 15:35:48.080682 6105 sys_catalog.cc:564] T 00000000000000000000000000000000 P 7443e0f995c14523bb438d06089c5035 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 15:35:48.080807 6110 consensus_queue.cc:237] T 00000000000000000000000000000000 P 7443e0f995c14523bb438d06089c5035 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7443e0f995c14523bb438d06089c5035" member_type: VOTER last_known_addr { host: "127.3.89.254" port: 42289 } }
I20260809 15:35:48.082405 6112 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7443e0f995c14523bb438d06089c5035 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 7443e0f995c14523bb438d06089c5035. Latest consensus state: current_term: 1 leader_uuid: "7443e0f995c14523bb438d06089c5035" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7443e0f995c14523bb438d06089c5035" member_type: VOTER last_known_addr { host: "127.3.89.254" port: 42289 } } }
I20260809 15:35:48.082520 6112 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7443e0f995c14523bb438d06089c5035 [sys.catalog]: This master's current role is: LEADER
I20260809 15:35:48.082945 6111 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7443e0f995c14523bb438d06089c5035 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "7443e0f995c14523bb438d06089c5035" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7443e0f995c14523bb438d06089c5035" member_type: VOTER last_known_addr { host: "127.3.89.254" port: 42289 } } }
I20260809 15:35:48.083077 6111 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7443e0f995c14523bb438d06089c5035 [sys.catalog]: This master's current role is: LEADER
W20260809 15:35:48.085435 6125 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 7443e0f995c14523bb438d06089c5035: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260809 15:35:48.085529 6125 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260809 15:35:48.085601 6122 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 15:35:48.086314 6122 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 15:35:48.091877 6122 catalog_manager.cc:1347] Generated new cluster ID: 7d45a2fc87be4a8ea59a576988667a42
I20260809 15:35:48.091951 6122 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 15:35:48.104158 6122 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 15:35:48.105094 6122 catalog_manager.cc:1504] Loading token signing keys...
I20260809 15:35:48.109887 6122 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 7443e0f995c14523bb438d06089c5035: Generated new TSK 0
I20260809 15:35:48.110661 6122 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.3.89.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.3.89.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.3.89.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786289732823074-3431-0/krb5kdc/kudu_127.3.89.193.keytab.
Entry for principal kudu/127.3.89.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786289732823074-3431-0/krb5kdc/kudu_127.3.89.193.keytab.
I20260809 15:35:48.146656 3431 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskm314YR/build/debug/bin/kudu
/tmp/dist-test-taskm314YR/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786289732823074-3431-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786289732823074-3431-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786289732823074-3431-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786289732823074-3431-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.3.89.193:0
--local_ip_for_outbound_sockets=127.3.89.193
--webserver_interface=127.3.89.193
--webserver_port=0
--tserver_master_addrs=127.3.89.254:42289
--keytab_file=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786289732823074-3431-0/krb5kdc/kudu_127.3.89.193.keytab
--principal=kudu/127.3.89.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.3.89.212:43231
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786289732823074-3431-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786289732823074-3431-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786289732823074-3431-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 15:35:48.257169 6132 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 15:35:48.257448 6132 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 15:35:48.257529 6132 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 15:35:48.261534 6132 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 15:35:48.261620 6132 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 15:35:48.261710 6132 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.3.89.193
I20260809 15:35:48.266783 6132 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.3.89.212:43231
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786289732823074-3431-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786289732823074-3431-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786289732823074-3431-0/krb5kdc/kudu_127.3.89.193.keytab
--principal=kudu/127.3.89.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.3.89.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786289732823074-3431-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.3.89.193
--webserver_port=0
--tserver_master_addrs=127.3.89.254:42289
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.6132
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.3.89.193
--log_dir=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786289732823074-3431-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 09 Aug 2026 14:46:55 UTC on ccycloud.quasar-yswkmf.root.comops.site
build id 4576540
I20260809 15:35:48.267802 6132 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 15:35:48.268808 6132 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 15:35:48.278131 6140 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 15:35:48.278105 6137 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 15:35:48.278112 6138 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 15:35:48.278460 6132 server_base.cc:1034] running on GCE node
I20260809 15:35:48.278923 6132 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 15:35:48.279546 6132 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 15:35:48.280735 6132 hybrid_clock.cc:648] HybridClock initialized: now 1786289748280721 us; error 32 us; skew 500 ppm
Aug 09 15:35:48 dist-test-slave-2w3w krb5kdc[6017](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786289748, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.89.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 15:35:48.283828 6132 init.cc:377] Logged in from keytab as kudu/127.3.89.193@KRBTEST.COM (short username kudu)
I20260809 15:35:48.284647 6132 webserver.cc:460] Webserver started at http://127.3.89.193:38055/ using document root <none> and password file <none>
I20260809 15:35:48.285194 6132 fs_manager.cc:359] Metadata directory not provided
I20260809 15:35:48.285302 6132 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 15:35:48.285527 6132 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 15:35:48.287572 6132 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786289732823074-3431-0/minicluster-data/ts-0/data/instance:
uuid: "d56cd2b7d6ef4846bfedafb4190c8b2b"
format_stamp: "Formatted at 2026-08-09 15:35:48 on dist-test-slave-2w3w"
server_key: "33a9612266b3e39f8b1599fea18b8982"
server_key_iv: "d1d40485777b44bbd8fbfade6ec6f59b"
server_key_version: "encryptionkey@0"
I20260809 15:35:48.288143 6132 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786289732823074-3431-0/minicluster-data/ts-0/wal/instance:
uuid: "d56cd2b7d6ef4846bfedafb4190c8b2b"
format_stamp: "Formatted at 2026-08-09 15:35:48 on dist-test-slave-2w3w"
server_key: "33a9612266b3e39f8b1599fea18b8982"
server_key_iv: "d1d40485777b44bbd8fbfade6ec6f59b"
server_key_version: "encryptionkey@0"
I20260809 15:35:48.291808 6132 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.004s sys 0.000s
I20260809 15:35:48.294124 6149 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 15:35:48.295425 6132 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260809 15:35:48.295558 6132 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786289732823074-3431-0/minicluster-data/ts-0/data,/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786289732823074-3431-0/minicluster-data/ts-0/wal
uuid: "d56cd2b7d6ef4846bfedafb4190c8b2b"
format_stamp: "Formatted at 2026-08-09 15:35:48 on dist-test-slave-2w3w"
server_key: "33a9612266b3e39f8b1599fea18b8982"
server_key_iv: "d1d40485777b44bbd8fbfade6ec6f59b"
server_key_version: "encryptionkey@0"
I20260809 15:35:48.295665 6132 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786289732823074-3431-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786289732823074-3431-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786289732823074-3431-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 15:35:48.316536 6132 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 15:35:48.319766 6132 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 15:35:48.319978 6132 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 15:35:48.320557 6132 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 15:35:48.321372 6132 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 15:35:48.321442 6132 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 15:35:48.321513 6132 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 15:35:48.321564 6132 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 15:35:48.330451 6132 rpc_server.cc:307] RPC server started. Bound to: 127.3.89.193:36947
I20260809 15:35:48.330524 6270 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.3.89.193:36947 every 8 connection(s)
I20260809 15:35:48.331429 6132 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786289732823074-3431-0/minicluster-data/ts-0/data/info.pb
I20260809 15:35:48.333259 3431 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskm314YR/build/debug/bin/kudu as pid 6132
I20260809 15:35:48.333403 3431 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786289732823074-3431-0/minicluster-data/ts-0/wal/instance
I20260809 15:35:48.333686 3431 external_mini_cluster.cc:1428] Setting key 19834b084c99c9b5a13fb3d48ba1a3a8
Aug 09 15:35:48 dist-test-slave-2w3w krb5kdc[6017](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786289748, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.89.193@KRBTEST.COM for kudu/127.3.89.254@KRBTEST.COM
WARNING: no policy specified for kudu/127.3.89.194@KRBTEST.COM; defaulting to no policy
I20260809 15:35:48.345149 6108 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:35:48.333186 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.89.193:55443 (local address 127.3.89.254:42289)
0809 15:35:48.333338 (+ 152us) server_negotiation.cc:206] Beginning negotiation
0809 15:35:48.333343 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 15:35:48.334358 (+ 1015us) server_negotiation.cc:407] Connection header received
0809 15:35:48.335791 (+ 1433us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 15:35:48.335797 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 15:35:48.335857 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 15:35:48.335947 (+ 90us) server_negotiation.cc:226] Negotiated authn=SASL
0809 15:35:48.337054 (+ 1107us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:35:48.337589 (+ 535us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:35:48.338728 (+ 1139us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:35:48.339031 (+ 303us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:35:48.339152 (+ 121us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:35:48.341592 (+ 2440us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 15:35:48.341595 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 15:35:48.341597 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 15:35:48.341620 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0809 15:35:48.343084 (+ 1464us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 15:35:48.343599 (+ 515us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 15:35:48.343603 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 15:35:48.343605 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 15:35:48.343692 (+ 87us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 15:35:48.344063 (+ 371us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 15:35:48.344066 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 15:35:48.344068 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 15:35:48.344249 (+ 181us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 15:35:48.344360 (+ 111us) server_negotiation.cc:1035] Waiting for connection context
0809 15:35:48.344840 (+ 480us) server_negotiation.cc:299] Negotiation successful
0809 15:35:48.344976 (+ 136us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":58}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.3.89.194@KRBTEST.COM" created.
I20260809 15:35:48.345937 6273 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:35:48.333707 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.89.254:42289 (local address 127.3.89.193:55443)
0809 15:35:48.334200 (+ 493us) negotiation.cc:107] Waiting for socket to connect
0809 15:35:48.334252 (+ 52us) client_negotiation.cc:174] Beginning negotiation
0809 15:35:48.335586 (+ 1334us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 15:35:48.336082 (+ 496us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 15:35:48.336088 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 15:35:48.336437 (+ 349us) client_negotiation.cc:189] Negotiated authn=SASL
0809 15:35:48.336890 (+ 453us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:35:48.336903 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:35:48.337720 (+ 817us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:35:48.337723 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:35:48.338592 (+ 869us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:35:48.338602 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:35:48.339190 (+ 588us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:35:48.339194 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:35:48.339361 (+ 167us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:35:48.339964 (+ 603us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 15:35:48.339980 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0809 15:35:48.341421 (+ 1441us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 15:35:48.343200 (+ 1779us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 15:35:48.343205 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 15:35:48.343218 (+ 13us) client_negotiation.cc:760] calling sasl_client_step()
0809 15:35:48.343502 (+ 284us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 15:35:48.343802 (+ 300us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 15:35:48.343806 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 15:35:48.343808 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 15:35:48.343949 (+ 141us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 15:35:48.344360 (+ 411us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 15:35:48.344367 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 15:35:48.344600 (+ 233us) client_negotiation.cc:769] Sending connection context
0809 15:35:48.344787 (+ 187us) client_negotiation.cc:240] Negotiation successful
0809 15:35:48.345103 (+ 316us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":296,"thread_start_us":139,"threads_started":1}
I20260809 15:35:48.347179 6271 heartbeater.cc:347] Connected to a master server at 127.3.89.254:42289
I20260809 15:35:48.347419 6271 heartbeater.cc:464] Registering TS with master...
I20260809 15:35:48.347882 6271 heartbeater.cc:511] Master 127.3.89.254:42289 requested a full tablet report, sending...
I20260809 15:35:48.349458 6068 ts_manager.cc:194] Registered new tserver with Master: d56cd2b7d6ef4846bfedafb4190c8b2b (127.3.89.193:36947)
I20260809 15:35:48.351164 6068 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.3.89.193@KRBTEST.COM'} at 127.3.89.193:55443
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.3.89.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786289732823074-3431-0/krb5kdc/kudu_127.3.89.194.keytab.
Entry for principal kudu/127.3.89.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786289732823074-3431-0/krb5kdc/kudu_127.3.89.194.keytab.
I20260809 15:35:48.361510 3431 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskm314YR/build/debug/bin/kudu
/tmp/dist-test-taskm314YR/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786289732823074-3431-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786289732823074-3431-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786289732823074-3431-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786289732823074-3431-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.3.89.194:0
--local_ip_for_outbound_sockets=127.3.89.194
--webserver_interface=127.3.89.194
--webserver_port=0
--tserver_master_addrs=127.3.89.254:42289
--keytab_file=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786289732823074-3431-0/krb5kdc/kudu_127.3.89.194.keytab
--principal=kudu/127.3.89.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.3.89.212:43231
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786289732823074-3431-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786289732823074-3431-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786289732823074-3431-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 15:35:48.472684 6276 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 15:35:48.473202 6276 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 15:35:48.473380 6276 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 15:35:48.478138 6276 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 15:35:48.478229 6276 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 15:35:48.478325 6276 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.3.89.194
I20260809 15:35:48.483816 6276 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.3.89.212:43231
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786289732823074-3431-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786289732823074-3431-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786289732823074-3431-0/krb5kdc/kudu_127.3.89.194.keytab
--principal=kudu/127.3.89.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.3.89.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786289732823074-3431-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.3.89.194
--webserver_port=0
--tserver_master_addrs=127.3.89.254:42289
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.6276
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.3.89.194
--log_dir=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786289732823074-3431-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 09 Aug 2026 14:46:55 UTC on ccycloud.quasar-yswkmf.root.comops.site
build id 4576540
I20260809 15:35:48.484827 6276 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 15:35:48.485813 6276 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 15:35:48.495343 6282 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 15:35:48.495349 6284 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 15:35:48.495455 6281 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 15:35:48.495738 6276 server_base.cc:1034] running on GCE node
I20260809 15:35:48.496140 6276 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 15:35:48.496791 6276 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 15:35:48.497977 6276 hybrid_clock.cc:648] HybridClock initialized: now 1786289748497960 us; error 26 us; skew 500 ppm
Aug 09 15:35:48 dist-test-slave-2w3w krb5kdc[6017](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786289748, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.89.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 15:35:48.501199 6276 init.cc:377] Logged in from keytab as kudu/127.3.89.194@KRBTEST.COM (short username kudu)
I20260809 15:35:48.502135 6276 webserver.cc:460] Webserver started at http://127.3.89.194:33339/ using document root <none> and password file <none>
I20260809 15:35:48.502686 6276 fs_manager.cc:359] Metadata directory not provided
I20260809 15:35:48.502748 6276 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 15:35:48.502933 6276 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 15:35:48.505117 6276 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786289732823074-3431-0/minicluster-data/ts-1/data/instance:
uuid: "85ea5b4c9d5d40f4bc5ef201d2a4a6cf"
format_stamp: "Formatted at 2026-08-09 15:35:48 on dist-test-slave-2w3w"
server_key: "3abc88f5860aa4319c5520ca0b0f828d"
server_key_iv: "188cbecf63e694a4bd76d8eab9337937"
server_key_version: "encryptionkey@0"
I20260809 15:35:48.505676 6276 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786289732823074-3431-0/minicluster-data/ts-1/wal/instance:
uuid: "85ea5b4c9d5d40f4bc5ef201d2a4a6cf"
format_stamp: "Formatted at 2026-08-09 15:35:48 on dist-test-slave-2w3w"
server_key: "3abc88f5860aa4319c5520ca0b0f828d"
server_key_iv: "188cbecf63e694a4bd76d8eab9337937"
server_key_version: "encryptionkey@0"
I20260809 15:35:48.509503 6276 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.002s
I20260809 15:35:48.512019 6293 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 15:35:48.513209 6276 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260809 15:35:48.513352 6276 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786289732823074-3431-0/minicluster-data/ts-1/data,/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786289732823074-3431-0/minicluster-data/ts-1/wal
uuid: "85ea5b4c9d5d40f4bc5ef201d2a4a6cf"
format_stamp: "Formatted at 2026-08-09 15:35:48 on dist-test-slave-2w3w"
server_key: "3abc88f5860aa4319c5520ca0b0f828d"
server_key_iv: "188cbecf63e694a4bd76d8eab9337937"
server_key_version: "encryptionkey@0"
I20260809 15:35:48.513463 6276 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786289732823074-3431-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786289732823074-3431-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786289732823074-3431-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 15:35:48.536947 6276 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 15:35:48.540248 6276 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 15:35:48.540468 6276 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 15:35:48.541088 6276 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 15:35:48.541949 6276 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 15:35:48.541999 6276 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 15:35:48.542083 6276 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 15:35:48.542127 6276 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 15:35:48.551390 6276 rpc_server.cc:307] RPC server started. Bound to: 127.3.89.194:43443
I20260809 15:35:48.551414 6417 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.3.89.194:43443 every 8 connection(s)
I20260809 15:35:48.552287 6276 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786289732823074-3431-0/minicluster-data/ts-1/data/info.pb
I20260809 15:35:48.557960 3431 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskm314YR/build/debug/bin/kudu as pid 6276
I20260809 15:35:48.558089 3431 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786289732823074-3431-0/minicluster-data/ts-1/wal/instance
I20260809 15:35:48.558415 3431 external_mini_cluster.cc:1428] Setting key 1096a2dfac208e1bb67f0ae02125a8a7
Aug 09 15:35:48 dist-test-slave-2w3w krb5kdc[6017](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786289748, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.89.194@KRBTEST.COM for kudu/127.3.89.254@KRBTEST.COM
I20260809 15:35:48.567061 6108 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:35:48.554143 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.89.194:55353 (local address 127.3.89.254:42289)
0809 15:35:48.554336 (+ 193us) server_negotiation.cc:206] Beginning negotiation
0809 15:35:48.554343 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 15:35:48.555382 (+ 1039us) server_negotiation.cc:407] Connection header received
0809 15:35:48.556817 (+ 1435us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 15:35:48.556821 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 15:35:48.556895 (+ 74us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 15:35:48.556997 (+ 102us) server_negotiation.cc:226] Negotiated authn=SASL
0809 15:35:48.558221 (+ 1224us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:35:48.558965 (+ 744us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:35:48.560317 (+ 1352us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:35:48.560686 (+ 369us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:35:48.560825 (+ 139us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:35:48.563498 (+ 2673us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 15:35:48.563502 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 15:35:48.563503 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 15:35:48.563526 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0809 15:35:48.565042 (+ 1516us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 15:35:48.565557 (+ 515us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 15:35:48.565565 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 15:35:48.565566 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 15:35:48.565640 (+ 74us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 15:35:48.565984 (+ 344us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 15:35:48.565987 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 15:35:48.565988 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 15:35:48.566161 (+ 173us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 15:35:48.566264 (+ 103us) server_negotiation.cc:1035] Waiting for connection context
0809 15:35:48.566746 (+ 482us) server_negotiation.cc:299] Negotiation successful
0809 15:35:48.566875 (+ 129us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":51}
I20260809 15:35:48.567889 6420 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:35:48.554668 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.89.254:42289 (local address 127.3.89.194:55353)
0809 15:35:48.555212 (+ 544us) negotiation.cc:107] Waiting for socket to connect
0809 15:35:48.555256 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0809 15:35:48.556585 (+ 1329us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 15:35:48.557174 (+ 589us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 15:35:48.557181 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 15:35:48.557548 (+ 367us) client_negotiation.cc:189] Negotiated authn=SASL
0809 15:35:48.558046 (+ 498us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:35:48.558059 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:35:48.559108 (+ 1049us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:35:48.559111 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:35:48.560185 (+ 1074us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:35:48.560195 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:35:48.560837 (+ 642us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:35:48.560840 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:35:48.560980 (+ 140us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:35:48.561594 (+ 614us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 15:35:48.561609 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0809 15:35:48.563325 (+ 1716us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 15:35:48.565159 (+ 1834us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 15:35:48.565164 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 15:35:48.565177 (+ 13us) client_negotiation.cc:760] calling sasl_client_step()
0809 15:35:48.565443 (+ 266us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 15:35:48.565733 (+ 290us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 15:35:48.565737 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 15:35:48.565739 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 15:35:48.565877 (+ 138us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 15:35:48.566296 (+ 419us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 15:35:48.566302 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 15:35:48.566521 (+ 219us) client_negotiation.cc:769] Sending connection context
0809 15:35:48.566702 (+ 181us) client_negotiation.cc:240] Negotiation successful
0809 15:35:48.567022 (+ 320us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":347,"thread_start_us":121,"threads_started":1}
WARNING: no policy specified for kudu/127.3.89.195@KRBTEST.COM; defaulting to no policy
I20260809 15:35:48.569110 6418 heartbeater.cc:347] Connected to a master server at 127.3.89.254:42289
I20260809 15:35:48.569348 6418 heartbeater.cc:464] Registering TS with master...
I20260809 15:35:48.569840 6418 heartbeater.cc:511] Master 127.3.89.254:42289 requested a full tablet report, sending...
I20260809 15:35:48.570897 6068 ts_manager.cc:194] Registered new tserver with Master: 85ea5b4c9d5d40f4bc5ef201d2a4a6cf (127.3.89.194:43443)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.3.89.195@KRBTEST.COM" created.
I20260809 15:35:48.571663 6068 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.3.89.194@KRBTEST.COM'} at 127.3.89.194:55353
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.3.89.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786289732823074-3431-0/krb5kdc/kudu_127.3.89.195.keytab.
Entry for principal kudu/127.3.89.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786289732823074-3431-0/krb5kdc/kudu_127.3.89.195.keytab.
I20260809 15:35:48.586023 3431 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskm314YR/build/debug/bin/kudu
/tmp/dist-test-taskm314YR/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786289732823074-3431-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786289732823074-3431-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786289732823074-3431-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786289732823074-3431-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.3.89.195:0
--local_ip_for_outbound_sockets=127.3.89.195
--webserver_interface=127.3.89.195
--webserver_port=0
--tserver_master_addrs=127.3.89.254:42289
--keytab_file=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786289732823074-3431-0/krb5kdc/kudu_127.3.89.195.keytab
--principal=kudu/127.3.89.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.3.89.212:43231
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786289732823074-3431-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786289732823074-3431-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786289732823074-3431-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 15:35:48.701354 6423 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 15:35:48.701627 6423 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 15:35:48.701700 6423 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 15:35:48.705753 6423 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 15:35:48.705834 6423 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 15:35:48.705915 6423 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.3.89.195
I20260809 15:35:48.711182 6423 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.3.89.212:43231
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786289732823074-3431-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786289732823074-3431-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786289732823074-3431-0/krb5kdc/kudu_127.3.89.195.keytab
--principal=kudu/127.3.89.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.3.89.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786289732823074-3431-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.3.89.195
--webserver_port=0
--tserver_master_addrs=127.3.89.254:42289
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.6423
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.3.89.195
--log_dir=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786289732823074-3431-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 09 Aug 2026 14:46:55 UTC on ccycloud.quasar-yswkmf.root.comops.site
build id 4576540
I20260809 15:35:48.712250 6423 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 15:35:48.713240 6423 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260809 15:35:48.723111 6423 server_base.cc:1034] running on GCE node
W20260809 15:35:48.723065 6429 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 15:35:48.723099 6431 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 15:35:48.723066 6428 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 15:35:48.723811 6423 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 15:35:48.724364 6423 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 15:35:48.725549 6423 hybrid_clock.cc:648] HybridClock initialized: now 1786289748725539 us; error 33 us; skew 500 ppm
Aug 09 15:35:48 dist-test-slave-2w3w krb5kdc[6017](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786289748, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.89.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 15:35:48.728531 6423 init.cc:377] Logged in from keytab as kudu/127.3.89.195@KRBTEST.COM (short username kudu)
I20260809 15:35:48.729314 6423 webserver.cc:460] Webserver started at http://127.3.89.195:41615/ using document root <none> and password file <none>
I20260809 15:35:48.729808 6423 fs_manager.cc:359] Metadata directory not provided
I20260809 15:35:48.729867 6423 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 15:35:48.730034 6423 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 15:35:48.732131 6423 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786289732823074-3431-0/minicluster-data/ts-2/data/instance:
uuid: "c3647e79f53045c6880f8c7c30c136ae"
format_stamp: "Formatted at 2026-08-09 15:35:48 on dist-test-slave-2w3w"
server_key: "acc7183d5e65456fe4bc478b45d7c5d3"
server_key_iv: "8276f12de155aefdc28d5aa3d62886a1"
server_key_version: "encryptionkey@0"
I20260809 15:35:48.732663 6423 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786289732823074-3431-0/minicluster-data/ts-2/wal/instance:
uuid: "c3647e79f53045c6880f8c7c30c136ae"
format_stamp: "Formatted at 2026-08-09 15:35:48 on dist-test-slave-2w3w"
server_key: "acc7183d5e65456fe4bc478b45d7c5d3"
server_key_iv: "8276f12de155aefdc28d5aa3d62886a1"
server_key_version: "encryptionkey@0"
I20260809 15:35:48.736200 6423 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.004s sys 0.000s
I20260809 15:35:48.738413 6438 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 15:35:48.739655 6423 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260809 15:35:48.739754 6423 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786289732823074-3431-0/minicluster-data/ts-2/data,/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786289732823074-3431-0/minicluster-data/ts-2/wal
uuid: "c3647e79f53045c6880f8c7c30c136ae"
format_stamp: "Formatted at 2026-08-09 15:35:48 on dist-test-slave-2w3w"
server_key: "acc7183d5e65456fe4bc478b45d7c5d3"
server_key_iv: "8276f12de155aefdc28d5aa3d62886a1"
server_key_version: "encryptionkey@0"
I20260809 15:35:48.739861 6423 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786289732823074-3431-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786289732823074-3431-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786289732823074-3431-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 15:35:48.751091 6423 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 15:35:48.754194 6423 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 15:35:48.754338 6423 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 15:35:48.754833 6423 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 15:35:48.755614 6423 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 15:35:48.755661 6423 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 15:35:48.755697 6423 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 15:35:48.755720 6423 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 15:35:48.764479 6423 rpc_server.cc:307] RPC server started. Bound to: 127.3.89.195:36533
I20260809 15:35:48.764515 6564 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.3.89.195:36533 every 8 connection(s)
I20260809 15:35:48.765352 6423 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786289732823074-3431-0/minicluster-data/ts-2/data/info.pb
I20260809 15:35:48.772536 3431 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskm314YR/build/debug/bin/kudu as pid 6423
I20260809 15:35:48.772656 3431 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786289732823074-3431-0/minicluster-data/ts-2/wal/instance
I20260809 15:35:48.773015 3431 external_mini_cluster.cc:1428] Setting key 86ed3217744f6f45ce966da16ffdeff9
Aug 09 15:35:48 dist-test-slave-2w3w krb5kdc[6017](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786289748, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.89.195@KRBTEST.COM for kudu/127.3.89.254@KRBTEST.COM
I20260809 15:35:48.778504 6108 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:35:48.767086 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.89.195:41365 (local address 127.3.89.254:42289)
0809 15:35:48.767225 (+ 139us) server_negotiation.cc:206] Beginning negotiation
0809 15:35:48.767229 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 15:35:48.768091 (+ 862us) server_negotiation.cc:407] Connection header received
0809 15:35:48.769327 (+ 1236us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 15:35:48.769330 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 15:35:48.769378 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 15:35:48.769453 (+ 75us) server_negotiation.cc:226] Negotiated authn=SASL
0809 15:35:48.770483 (+ 1030us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:35:48.771032 (+ 549us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:35:48.772112 (+ 1080us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:35:48.772328 (+ 216us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:35:48.772402 (+ 74us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:35:48.775322 (+ 2920us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 15:35:48.775324 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 15:35:48.775327 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 15:35:48.775350 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0809 15:35:48.776829 (+ 1479us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 15:35:48.777257 (+ 428us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 15:35:48.777260 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 15:35:48.777261 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 15:35:48.777354 (+ 93us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 15:35:48.777633 (+ 279us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 15:35:48.777636 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 15:35:48.777637 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 15:35:48.777797 (+ 160us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 15:35:48.777873 (+ 76us) server_negotiation.cc:1035] Waiting for connection context
0809 15:35:48.778282 (+ 409us) server_negotiation.cc:299] Negotiation successful
0809 15:35:48.778377 (+ 95us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":52}
I20260809 15:35:48.779382 6567 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:35:48.767455 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.89.254:42289 (local address 127.3.89.195:41365)
0809 15:35:48.767927 (+ 472us) negotiation.cc:107] Waiting for socket to connect
0809 15:35:48.767970 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0809 15:35:48.769132 (+ 1162us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 15:35:48.769589 (+ 457us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 15:35:48.769595 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 15:35:48.769916 (+ 321us) client_negotiation.cc:189] Negotiated authn=SASL
0809 15:35:48.770350 (+ 434us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:35:48.770364 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:35:48.771137 (+ 773us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:35:48.771140 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:35:48.772002 (+ 862us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:35:48.772011 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:35:48.772476 (+ 465us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:35:48.772481 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:35:48.772694 (+ 213us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:35:48.773336 (+ 642us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 15:35:48.773361 (+ 25us) client_negotiation.cc:656] calling sasl_client_start()
0809 15:35:48.775185 (+ 1824us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 15:35:48.776927 (+ 1742us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 15:35:48.776931 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 15:35:48.776941 (+ 10us) client_negotiation.cc:760] calling sasl_client_step()
0809 15:35:48.777174 (+ 233us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 15:35:48.777432 (+ 258us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 15:35:48.777434 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 15:35:48.777436 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 15:35:48.777557 (+ 121us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 15:35:48.777883 (+ 326us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 15:35:48.777888 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 15:35:48.778087 (+ 199us) client_negotiation.cc:769] Sending connection context
0809 15:35:48.778235 (+ 148us) client_negotiation.cc:240] Negotiation successful
0809 15:35:48.778472 (+ 237us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":270,"thread_start_us":130,"threads_started":1}
I20260809 15:35:48.780488 6565 heartbeater.cc:347] Connected to a master server at 127.3.89.254:42289
I20260809 15:35:48.780715 6565 heartbeater.cc:464] Registering TS with master...
I20260809 15:35:48.781229 6565 heartbeater.cc:511] Master 127.3.89.254:42289 requested a full tablet report, sending...
I20260809 15:35:48.782297 6068 ts_manager.cc:194] Registered new tserver with Master: c3647e79f53045c6880f8c7c30c136ae (127.3.89.195:36533)
I20260809 15:35:48.782855 6068 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.3.89.195@KRBTEST.COM'} at 127.3.89.195:41365
I20260809 15:35:48.786775 3431 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
W20260809 15:35:48.800887 6108 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 15:35:48.799156 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55302 (local address 127.3.89.254:42289)
0809 15:35:48.799268 (+ 112us) server_negotiation.cc:206] Beginning negotiation
0809 15:35:48.799272 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 15:35:48.799540 (+ 268us) server_negotiation.cc:407] Connection header received
0809 15:35:48.799724 (+ 184us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 15:35:48.799727 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 15:35:48.799824 (+ 97us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 15:35:48.799901 (+ 77us) server_negotiation.cc:226] Negotiated authn=SASL
0809 15:35:48.800681 (+ 780us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:55302: BlockingRecv error: recv got EOF from 127.0.0.1:55302 (error 108)
Metrics: {"server-negotiator.queue_time_us":32}
I20260809 15:35:48.801762 3431 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskm314YR/build/debug/bin/kudu with pid 6132
I20260809 15:35:48.809316 3431 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskm314YR/build/debug/bin/kudu with pid 6276
I20260809 15:35:48.815953 3431 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskm314YR/build/debug/bin/kudu with pid 6423
I20260809 15:35:48.821727 3431 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskm314YR/build/debug/bin/kudu with pid 6031
2026-08-09T15:35:48Z chronyd exiting
[ OK ] SecurityITest.TestNoKerberosCredentials (2141 ms)
[ RUN ] SecurityITest.SaslPlainFallback
Loading random data
Initializing database '/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786289732823074-3431-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 15:35:48 dist-test-slave-2w3w krb5kdc[6577](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 15:35:48 dist-test-slave-2w3w krb5kdc[6577](info): set up 2 sockets
Aug 09 15:35:48 dist-test-slave-2w3w krb5kdc[6577](info): commencing operation
krb5kdc: starting...
W20260809 15:35:49.900552 3431 mini_kdc.cc:121] Time spent starting KDC: real 1.056s user 0.000s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786289732823074-3431-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786289732823074-3431-0/krb5kdc/test-user.keytab.
Aug 09 15:35:49 dist-test-slave-2w3w krb5kdc[6577](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786289749, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T15:35:49Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T15:35:49Z Disabled control of system clock
WARNING: no policy specified for kudu/127.3.89.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.3.89.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.3.89.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786289732823074-3431-0/krb5kdc/kudu_127.3.89.254.keytab.
Entry for principal kudu/127.3.89.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786289732823074-3431-0/krb5kdc/kudu_127.3.89.254.keytab.
I20260809 15:35:50.025861 3431 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskm314YR/build/debug/bin/kudu
/tmp/dist-test-taskm314YR/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786289732823074-3431-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786289732823074-3431-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786289732823074-3431-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786289732823074-3431-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.3.89.254:37599
--webserver_interface=127.3.89.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786289732823074-3431-0/krb5kdc/kudu_127.3.89.254.keytab
--principal=kudu/127.3.89.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.3.89.212:35501
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.3.89.254:37599
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc-authentication=optional
--user-acl=* with env {KRB5CCNAME=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786289732823074-3431-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786289732823074-3431-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786289732823074-3431-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 15:35:50.134044 6591 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 15:35:50.134326 6591 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 15:35:50.134399 6591 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 15:35:50.138397 6591 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 15:35:50.138484 6591 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 15:35:50.138522 6591 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 15:35:50.138579 6591 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 15:35:50.138619 6591 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 15:35:50.143646 6591 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.3.89.212:35501
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786289732823074-3431-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786289732823074-3431-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.3.89.254:37599
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786289732823074-3431-0/krb5kdc/kudu_127.3.89.254.keytab
--principal=kudu/127.3.89.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.3.89.254:37599
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786289732823074-3431-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.3.89.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.6591
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786289732823074-3431-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 09 Aug 2026 14:46:55 UTC on ccycloud.quasar-yswkmf.root.comops.site
build id 4576540
I20260809 15:35:50.144614 6591 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 15:35:50.145561 6591 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 15:35:50.153848 6597 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 15:35:50.153857 6596 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 15:35:50.154045 6591 server_base.cc:1034] running on GCE node
W20260809 15:35:50.153848 6600 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 15:35:50.154752 6591 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 15:35:50.155911 6591 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 15:35:50.157094 6591 hybrid_clock.cc:648] HybridClock initialized: now 1786289750157069 us; error 40 us; skew 500 ppm
Aug 09 15:35:50 dist-test-slave-2w3w krb5kdc[6577](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786289750, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.89.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 15:35:50.159842 6591 init.cc:377] Logged in from keytab as kudu/127.3.89.254@KRBTEST.COM (short username kudu)
I20260809 15:35:50.160717 6591 webserver.cc:460] Webserver started at http://127.3.89.254:45413/ using document root <none> and password file <none>
I20260809 15:35:50.161247 6591 fs_manager.cc:359] Metadata directory not provided
I20260809 15:35:50.161345 6591 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 15:35:50.161574 6591 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 15:35:50.163615 6591 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786289732823074-3431-0/minicluster-data/master-0/data/instance:
uuid: "6500bcd05602435d8d6c7c42446c512b"
format_stamp: "Formatted at 2026-08-09 15:35:50 on dist-test-slave-2w3w"
server_key: "ba63e2c6a45ba10a6815740e3839ace7"
server_key_iv: "77c0293f4cdd3abb44d68d3e38b65585"
server_key_version: "encryptionkey@0"
I20260809 15:35:50.164207 6591 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786289732823074-3431-0/minicluster-data/master-0/wal/instance:
uuid: "6500bcd05602435d8d6c7c42446c512b"
format_stamp: "Formatted at 2026-08-09 15:35:50 on dist-test-slave-2w3w"
server_key: "ba63e2c6a45ba10a6815740e3839ace7"
server_key_iv: "77c0293f4cdd3abb44d68d3e38b65585"
server_key_version: "encryptionkey@0"
I20260809 15:35:50.167744 6591 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.003s sys 0.000s
I20260809 15:35:50.169907 6608 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 15:35:50.171097 6591 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260809 15:35:50.171236 6591 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786289732823074-3431-0/minicluster-data/master-0/data,/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786289732823074-3431-0/minicluster-data/master-0/wal
uuid: "6500bcd05602435d8d6c7c42446c512b"
format_stamp: "Formatted at 2026-08-09 15:35:50 on dist-test-slave-2w3w"
server_key: "ba63e2c6a45ba10a6815740e3839ace7"
server_key_iv: "77c0293f4cdd3abb44d68d3e38b65585"
server_key_version: "encryptionkey@0"
I20260809 15:35:50.171358 6591 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786289732823074-3431-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786289732823074-3431-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786289732823074-3431-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 15:35:50.190779 6591 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 15:35:50.193941 6591 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 15:35:50.194139 6591 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 15:35:50.201062 6591 rpc_server.cc:307] RPC server started. Bound to: 127.3.89.254:37599
I20260809 15:35:50.201449 6661 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.3.89.254:37599 every 8 connection(s)
I20260809 15:35:50.201999 6591 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786289732823074-3431-0/minicluster-data/master-0/data/info.pb
I20260809 15:35:50.204674 6663 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 15:35:50.210193 6663 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6500bcd05602435d8d6c7c42446c512b: Bootstrap starting.
I20260809 15:35:50.211755 3431 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskm314YR/build/debug/bin/kudu as pid 6591
I20260809 15:35:50.211884 3431 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786289732823074-3431-0/minicluster-data/master-0/wal/instance
I20260809 15:35:50.212174 3431 external_mini_cluster.cc:1428] Setting key 9049c8ec8e718b20423f5e24121386cd
I20260809 15:35:50.212924 6663 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 6500bcd05602435d8d6c7c42446c512b: Neither blocks nor log segments found. Creating new log.
I20260809 15:35:50.213816 6663 log.cc:824] T 00000000000000000000000000000000 P 6500bcd05602435d8d6c7c42446c512b: Log is configured to *not* fsync() on all Append() calls
I20260809 15:35:50.216012 6663 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6500bcd05602435d8d6c7c42446c512b: No bootstrap required, opened a new log
I20260809 15:35:50.220237 6663 raft_consensus.cc:348] T 00000000000000000000000000000000 P 6500bcd05602435d8d6c7c42446c512b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6500bcd05602435d8d6c7c42446c512b" member_type: VOTER last_known_addr { host: "127.3.89.254" port: 37599 } }
I20260809 15:35:50.220533 6663 raft_consensus.cc:374] T 00000000000000000000000000000000 P 6500bcd05602435d8d6c7c42446c512b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 15:35:50.220600 6663 raft_consensus.cc:729] T 00000000000000000000000000000000 P 6500bcd05602435d8d6c7c42446c512b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6500bcd05602435d8d6c7c42446c512b, State: Initialized, Role: FOLLOWER
I20260809 15:35:50.221081 6663 consensus_queue.cc:260] T 00000000000000000000000000000000 P 6500bcd05602435d8d6c7c42446c512b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6500bcd05602435d8d6c7c42446c512b" member_type: VOTER last_known_addr { host: "127.3.89.254" port: 37599 } }
I20260809 15:35:50.221210 6663 raft_consensus.cc:388] T 00000000000000000000000000000000 P 6500bcd05602435d8d6c7c42446c512b [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 15:35:50.221292 6663 raft_consensus.cc:482] T 00000000000000000000000000000000 P 6500bcd05602435d8d6c7c42446c512b [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 15:35:50.221383 6663 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 6500bcd05602435d8d6c7c42446c512b [term 0 FOLLOWER]: Advancing to term 1
Aug 09 15:35:50 dist-test-slave-2w3w krb5kdc[6577](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786289749, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.3.89.254@KRBTEST.COM
I20260809 15:35:50.222429 6663 raft_consensus.cc:504] T 00000000000000000000000000000000 P 6500bcd05602435d8d6c7c42446c512b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6500bcd05602435d8d6c7c42446c512b" member_type: VOTER last_known_addr { host: "127.3.89.254" port: 37599 } }
I20260809 15:35:50.222793 6663 leader_election.cc:302] T 00000000000000000000000000000000 P 6500bcd05602435d8d6c7c42446c512b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 6500bcd05602435d8d6c7c42446c512b; no voters:
I20260809 15:35:50.223178 6663 leader_election.cc:288] T 00000000000000000000000000000000 P 6500bcd05602435d8d6c7c42446c512b [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 15:35:50.223297 6669 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 6500bcd05602435d8d6c7c42446c512b [term 1 FOLLOWER]: Leader election won for term 1
I20260809 15:35:50.223551 6669 raft_consensus.cc:686] T 00000000000000000000000000000000 P 6500bcd05602435d8d6c7c42446c512b [term 1 LEADER]: Becoming Leader. State: Replica: 6500bcd05602435d8d6c7c42446c512b, State: Running, Role: LEADER
I20260809 15:35:50.223902 6669 consensus_queue.cc:237] T 00000000000000000000000000000000 P 6500bcd05602435d8d6c7c42446c512b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6500bcd05602435d8d6c7c42446c512b" member_type: VOTER last_known_addr { host: "127.3.89.254" port: 37599 } }
I20260809 15:35:50.224537 6663 sys_catalog.cc:564] T 00000000000000000000000000000000 P 6500bcd05602435d8d6c7c42446c512b [sys.catalog]: configured and running, proceeding with master startup.
I20260809 15:35:50.225436 6670 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6500bcd05602435d8d6c7c42446c512b [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "6500bcd05602435d8d6c7c42446c512b" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6500bcd05602435d8d6c7c42446c512b" member_type: VOTER last_known_addr { host: "127.3.89.254" port: 37599 } } }
I20260809 15:35:50.225636 6670 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6500bcd05602435d8d6c7c42446c512b [sys.catalog]: This master's current role is: LEADER
I20260809 15:35:50.226109 6681 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 15:35:50.226985 6671 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6500bcd05602435d8d6c7c42446c512b [sys.catalog]: SysCatalogTable state changed. Reason: New leader 6500bcd05602435d8d6c7c42446c512b. Latest consensus state: current_term: 1 leader_uuid: "6500bcd05602435d8d6c7c42446c512b" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6500bcd05602435d8d6c7c42446c512b" member_type: VOTER last_known_addr { host: "127.3.89.254" port: 37599 } } }
I20260809 15:35:50.227099 6671 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6500bcd05602435d8d6c7c42446c512b [sys.catalog]: This master's current role is: LEADER
I20260809 15:35:50.228296 6666 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:35:50.213261 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47652 (local address 127.3.89.254:37599)
0809 15:35:50.213701 (+ 440us) server_negotiation.cc:206] Beginning negotiation
0809 15:35:50.213751 (+ 50us) server_negotiation.cc:399] Waiting for connection header
0809 15:35:50.213786 (+ 35us) server_negotiation.cc:407] Connection header received
0809 15:35:50.214645 (+ 859us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 15:35:50.214663 (+ 18us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 15:35:50.215049 (+ 386us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 15:35:50.215330 (+ 281us) server_negotiation.cc:226] Negotiated authn=SASL
0809 15:35:50.216262 (+ 932us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:35:50.217235 (+ 973us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:35:50.218469 (+ 1234us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:35:50.218940 (+ 471us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:35:50.219132 (+ 192us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:35:50.222180 (+ 3048us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 15:35:50.222186 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 15:35:50.222201 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 15:35:50.222234 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0809 15:35:50.224371 (+ 2137us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 15:35:50.224933 (+ 562us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 15:35:50.224941 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 15:35:50.224946 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0809 15:35:50.225082 (+ 136us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 15:35:50.225451 (+ 369us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 15:35:50.225457 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 15:35:50.225459 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 15:35:50.225986 (+ 527us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 15:35:50.226188 (+ 202us) server_negotiation.cc:1035] Waiting for connection context
0809 15:35:50.227019 (+ 831us) server_negotiation.cc:299] Negotiation successful
0809 15:35:50.227378 (+ 359us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":294,"thread_start_us":137,"threads_started":1}
I20260809 15:35:50.229753 6681 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 15:35:50.231283 3431 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260809 15:35:50.236136 6681 catalog_manager.cc:1347] Generated new cluster ID: c22e70ff13b7460f8d6c19556499f5f9
I20260809 15:35:50.236198 6681 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 15:35:50.245908 6681 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 15:35:50.246753 6681 catalog_manager.cc:1504] Loading token signing keys...
I20260809 15:35:50.250836 6681 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 6500bcd05602435d8d6c7c42446c512b: Generated new TSK 0
I20260809 15:35:50.251549 6681 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260809 15:35:50.258661 6666 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:35:50.253874 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47666 (local address 127.3.89.254:37599)
0809 15:35:50.254003 (+ 129us) server_negotiation.cc:206] Beginning negotiation
0809 15:35:50.254009 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 15:35:50.254257 (+ 248us) server_negotiation.cc:407] Connection header received
0809 15:35:50.254423 (+ 166us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 15:35:50.254426 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 15:35:50.254475 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 15:35:50.254558 (+ 83us) server_negotiation.cc:226] Negotiated authn=SASL
0809 15:35:50.255152 (+ 594us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:35:50.255673 (+ 521us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:35:50.256549 (+ 876us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:35:50.256796 (+ 247us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:35:50.256918 (+ 122us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:35:50.257764 (+ 846us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 15:35:50.257768 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 15:35:50.257771 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 15:35:50.258225 (+ 454us) server_negotiation.cc:922] calling sasl_server_start()
0809 15:35:50.258253 (+ 28us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 15:35:50.258269 (+ 16us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 15:35:50.258369 (+ 100us) server_negotiation.cc:1035] Waiting for connection context
0809 15:35:50.258461 (+ 92us) server_negotiation.cc:299] Negotiation successful
0809 15:35:50.258520 (+ 59us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":51}
I20260809 15:35:50.263124 3431 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskm314YR/build/debug/bin/kudu with pid 6591
2026-08-09T15:35:50Z chronyd exiting
[ OK ] SecurityITest.SaslPlainFallback (1439 ms)
[ RUN ] SecurityITest.TestUnauthorizedClientKerberosCredentials
Loading random data
Initializing database '/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786289732823074-3431-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 15:35:50 dist-test-slave-2w3w krb5kdc[6701](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 15:35:50 dist-test-slave-2w3w krb5kdc[6701](info): set up 2 sockets
Aug 09 15:35:50 dist-test-slave-2w3w krb5kdc[6701](info): commencing operation
krb5kdc: starting...
W20260809 15:35:51.348305 3431 mini_kdc.cc:121] Time spent starting KDC: real 1.064s user 0.000s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786289732823074-3431-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786289732823074-3431-0/krb5kdc/test-user.keytab.
Aug 09 15:35:51 dist-test-slave-2w3w krb5kdc[6701](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786289751, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T15:35:51Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T15:35:51Z Disabled control of system clock
WARNING: no policy specified for kudu/127.3.89.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.3.89.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.3.89.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786289732823074-3431-0/krb5kdc/kudu_127.3.89.254.keytab.
Entry for principal kudu/127.3.89.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786289732823074-3431-0/krb5kdc/kudu_127.3.89.254.keytab.
I20260809 15:35:51.473627 3431 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskm314YR/build/debug/bin/kudu
/tmp/dist-test-taskm314YR/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786289732823074-3431-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786289732823074-3431-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786289732823074-3431-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786289732823074-3431-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.3.89.254:44261
--webserver_interface=127.3.89.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786289732823074-3431-0/krb5kdc/kudu_127.3.89.254.keytab
--principal=kudu/127.3.89.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.3.89.212:45083
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.3.89.254:44261
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786289732823074-3431-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786289732823074-3431-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786289732823074-3431-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 15:35:51.584739 6715 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 15:35:51.584986 6715 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 15:35:51.585024 6715 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 15:35:51.588898 6715 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 15:35:51.588940 6715 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 15:35:51.588958 6715 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 15:35:51.588974 6715 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 15:35:51.588989 6715 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 15:35:51.594091 6715 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.3.89.212:45083
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786289732823074-3431-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786289732823074-3431-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.3.89.254:44261
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786289732823074-3431-0/krb5kdc/kudu_127.3.89.254.keytab
--principal=kudu/127.3.89.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.3.89.254:44261
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786289732823074-3431-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.3.89.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.6715
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786289732823074-3431-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 09 Aug 2026 14:46:55 UTC on ccycloud.quasar-yswkmf.root.comops.site
build id 4576540
I20260809 15:35:51.595114 6715 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 15:35:51.596107 6715 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 15:35:51.604449 6723 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 15:35:51.604449 6721 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 15:35:51.604653 6715 server_base.cc:1034] running on GCE node
W20260809 15:35:51.604451 6720 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 15:35:51.605316 6715 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 15:35:51.606375 6715 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 15:35:51.607584 6715 hybrid_clock.cc:648] HybridClock initialized: now 1786289751607566 us; error 30 us; skew 500 ppm
Aug 09 15:35:51 dist-test-slave-2w3w krb5kdc[6701](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786289751, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.89.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 15:35:51.610351 6715 init.cc:377] Logged in from keytab as kudu/127.3.89.254@KRBTEST.COM (short username kudu)
I20260809 15:35:51.611328 6715 webserver.cc:460] Webserver started at http://127.3.89.254:36709/ using document root <none> and password file <none>
I20260809 15:35:51.611917 6715 fs_manager.cc:359] Metadata directory not provided
I20260809 15:35:51.612018 6715 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 15:35:51.612258 6715 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 15:35:51.614349 6715 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786289732823074-3431-0/minicluster-data/master-0/data/instance:
uuid: "19d6a7e95df042d8bd616e3745b1a32d"
format_stamp: "Formatted at 2026-08-09 15:35:51 on dist-test-slave-2w3w"
server_key: "60819a2face91515b287ab014cc46867"
server_key_iv: "486687cca520ef896587d15c2acbe343"
server_key_version: "encryptionkey@0"
I20260809 15:35:51.614933 6715 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786289732823074-3431-0/minicluster-data/master-0/wal/instance:
uuid: "19d6a7e95df042d8bd616e3745b1a32d"
format_stamp: "Formatted at 2026-08-09 15:35:51 on dist-test-slave-2w3w"
server_key: "60819a2face91515b287ab014cc46867"
server_key_iv: "486687cca520ef896587d15c2acbe343"
server_key_version: "encryptionkey@0"
I20260809 15:35:51.619151 6715 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.000s
I20260809 15:35:51.621549 6731 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 15:35:51.622713 6715 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260809 15:35:51.622866 6715 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786289732823074-3431-0/minicluster-data/master-0/data,/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786289732823074-3431-0/minicluster-data/master-0/wal
uuid: "19d6a7e95df042d8bd616e3745b1a32d"
format_stamp: "Formatted at 2026-08-09 15:35:51 on dist-test-slave-2w3w"
server_key: "60819a2face91515b287ab014cc46867"
server_key_iv: "486687cca520ef896587d15c2acbe343"
server_key_version: "encryptionkey@0"
I20260809 15:35:51.623008 6715 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786289732823074-3431-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786289732823074-3431-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786289732823074-3431-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 15:35:51.640419 6715 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 15:35:51.643293 6715 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 15:35:51.643493 6715 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 15:35:51.650748 6715 rpc_server.cc:307] RPC server started. Bound to: 127.3.89.254:44261
I20260809 15:35:51.651110 6784 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.3.89.254:44261 every 8 connection(s)
I20260809 15:35:51.651877 6715 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786289732823074-3431-0/minicluster-data/master-0/data/info.pb
I20260809 15:35:51.654670 6785 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 15:35:51.660228 3431 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskm314YR/build/debug/bin/kudu as pid 6715
I20260809 15:35:51.660225 6785 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 19d6a7e95df042d8bd616e3745b1a32d: Bootstrap starting.
I20260809 15:35:51.660362 3431 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786289732823074-3431-0/minicluster-data/master-0/wal/instance
I20260809 15:35:51.660670 3431 external_mini_cluster.cc:1428] Setting key 4aabb00586c33f3f98ad812b66ee424d
I20260809 15:35:51.662899 6785 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 19d6a7e95df042d8bd616e3745b1a32d: Neither blocks nor log segments found. Creating new log.
I20260809 15:35:51.663698 6785 log.cc:824] T 00000000000000000000000000000000 P 19d6a7e95df042d8bd616e3745b1a32d: Log is configured to *not* fsync() on all Append() calls
I20260809 15:35:51.665529 6785 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 19d6a7e95df042d8bd616e3745b1a32d: No bootstrap required, opened a new log
I20260809 15:35:51.669310 6785 raft_consensus.cc:348] T 00000000000000000000000000000000 P 19d6a7e95df042d8bd616e3745b1a32d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "19d6a7e95df042d8bd616e3745b1a32d" member_type: VOTER last_known_addr { host: "127.3.89.254" port: 44261 } }
Aug 09 15:35:51 dist-test-slave-2w3w krb5kdc[6701](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786289751, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.3.89.254@KRBTEST.COM
I20260809 15:35:51.669584 6785 raft_consensus.cc:374] T 00000000000000000000000000000000 P 19d6a7e95df042d8bd616e3745b1a32d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 15:35:51.669660 6785 raft_consensus.cc:729] T 00000000000000000000000000000000 P 19d6a7e95df042d8bd616e3745b1a32d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 19d6a7e95df042d8bd616e3745b1a32d, State: Initialized, Role: FOLLOWER
I20260809 15:35:51.670162 6785 consensus_queue.cc:260] T 00000000000000000000000000000000 P 19d6a7e95df042d8bd616e3745b1a32d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "19d6a7e95df042d8bd616e3745b1a32d" member_type: VOTER last_known_addr { host: "127.3.89.254" port: 44261 } }
I20260809 15:35:51.670320 6785 raft_consensus.cc:388] T 00000000000000000000000000000000 P 19d6a7e95df042d8bd616e3745b1a32d [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 15:35:51.670428 6785 raft_consensus.cc:482] T 00000000000000000000000000000000 P 19d6a7e95df042d8bd616e3745b1a32d [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 15:35:51.670576 6785 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 19d6a7e95df042d8bd616e3745b1a32d [term 0 FOLLOWER]: Advancing to term 1
I20260809 15:35:51.671674 6785 raft_consensus.cc:504] T 00000000000000000000000000000000 P 19d6a7e95df042d8bd616e3745b1a32d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "19d6a7e95df042d8bd616e3745b1a32d" member_type: VOTER last_known_addr { host: "127.3.89.254" port: 44261 } }
I20260809 15:35:51.672015 6785 leader_election.cc:302] T 00000000000000000000000000000000 P 19d6a7e95df042d8bd616e3745b1a32d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 19d6a7e95df042d8bd616e3745b1a32d; no voters:
I20260809 15:35:51.672302 6785 leader_election.cc:288] T 00000000000000000000000000000000 P 19d6a7e95df042d8bd616e3745b1a32d [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 15:35:51.672435 6791 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 19d6a7e95df042d8bd616e3745b1a32d [term 1 FOLLOWER]: Leader election won for term 1
I20260809 15:35:51.672658 6791 raft_consensus.cc:686] T 00000000000000000000000000000000 P 19d6a7e95df042d8bd616e3745b1a32d [term 1 LEADER]: Becoming Leader. State: Replica: 19d6a7e95df042d8bd616e3745b1a32d, State: Running, Role: LEADER
I20260809 15:35:51.673074 6791 consensus_queue.cc:237] T 00000000000000000000000000000000 P 19d6a7e95df042d8bd616e3745b1a32d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "19d6a7e95df042d8bd616e3745b1a32d" member_type: VOTER last_known_addr { host: "127.3.89.254" port: 44261 } }
I20260809 15:35:51.673645 6785 sys_catalog.cc:564] T 00000000000000000000000000000000 P 19d6a7e95df042d8bd616e3745b1a32d [sys.catalog]: configured and running, proceeding with master startup.
I20260809 15:35:51.675066 6792 sys_catalog.cc:455] T 00000000000000000000000000000000 P 19d6a7e95df042d8bd616e3745b1a32d [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "19d6a7e95df042d8bd616e3745b1a32d" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "19d6a7e95df042d8bd616e3745b1a32d" member_type: VOTER last_known_addr { host: "127.3.89.254" port: 44261 } } }
I20260809 15:35:51.675062 6793 sys_catalog.cc:455] T 00000000000000000000000000000000 P 19d6a7e95df042d8bd616e3745b1a32d [sys.catalog]: SysCatalogTable state changed. Reason: New leader 19d6a7e95df042d8bd616e3745b1a32d. Latest consensus state: current_term: 1 leader_uuid: "19d6a7e95df042d8bd616e3745b1a32d" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "19d6a7e95df042d8bd616e3745b1a32d" member_type: VOTER last_known_addr { host: "127.3.89.254" port: 44261 } } }
I20260809 15:35:51.675263 6793 sys_catalog.cc:458] T 00000000000000000000000000000000 P 19d6a7e95df042d8bd616e3745b1a32d [sys.catalog]: This master's current role is: LEADER
I20260809 15:35:51.675560 6792 sys_catalog.cc:458] T 00000000000000000000000000000000 P 19d6a7e95df042d8bd616e3745b1a32d [sys.catalog]: This master's current role is: LEADER
I20260809 15:35:51.675856 6801 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 15:35:51.676409 6789 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:35:51.662020 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33180 (local address 127.3.89.254:44261)
0809 15:35:51.662482 (+ 462us) server_negotiation.cc:206] Beginning negotiation
0809 15:35:51.662496 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0809 15:35:51.662532 (+ 36us) server_negotiation.cc:407] Connection header received
0809 15:35:51.663256 (+ 724us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 15:35:51.663275 (+ 19us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 15:35:51.663580 (+ 305us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 15:35:51.663952 (+ 372us) server_negotiation.cc:226] Negotiated authn=SASL
0809 15:35:51.664804 (+ 852us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:35:51.665839 (+ 1035us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:35:51.666699 (+ 860us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:35:51.667223 (+ 524us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:35:51.667392 (+ 169us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:35:51.670208 (+ 2816us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 15:35:51.670213 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 15:35:51.670225 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 15:35:51.670250 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0809 15:35:51.672501 (+ 2251us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 15:35:51.673034 (+ 533us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 15:35:51.673040 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 15:35:51.673045 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0809 15:35:51.673198 (+ 153us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 15:35:51.673521 (+ 323us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 15:35:51.673524 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 15:35:51.673526 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 15:35:51.674062 (+ 536us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 15:35:51.674263 (+ 201us) server_negotiation.cc:1035] Waiting for connection context
0809 15:35:51.675107 (+ 844us) server_negotiation.cc:299] Negotiation successful
0809 15:35:51.675424 (+ 317us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":324,"thread_start_us":134,"threads_started":1}
I20260809 15:35:51.679821 6801 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 15:35:51.685019 6801 catalog_manager.cc:1347] Generated new cluster ID: 51de92560f3e4a90a921a06b1a18f83c
I20260809 15:35:51.685117 6801 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 15:35:51.726636 6801 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 15:35:51.727837 6801 catalog_manager.cc:1504] Loading token signing keys...
I20260809 15:35:51.733881 6801 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 19d6a7e95df042d8bd616e3745b1a32d: Generated new TSK 0
I20260809 15:35:51.734632 6801 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.3.89.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.3.89.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.3.89.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786289732823074-3431-0/krb5kdc/kudu_127.3.89.193.keytab.
Entry for principal kudu/127.3.89.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786289732823074-3431-0/krb5kdc/kudu_127.3.89.193.keytab.
I20260809 15:35:51.775087 3431 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskm314YR/build/debug/bin/kudu
/tmp/dist-test-taskm314YR/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786289732823074-3431-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786289732823074-3431-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786289732823074-3431-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786289732823074-3431-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.3.89.193:0
--local_ip_for_outbound_sockets=127.3.89.193
--webserver_interface=127.3.89.193
--webserver_port=0
--tserver_master_addrs=127.3.89.254:44261
--keytab_file=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786289732823074-3431-0/krb5kdc/kudu_127.3.89.193.keytab
--principal=kudu/127.3.89.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.3.89.212:45083
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786289732823074-3431-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786289732823074-3431-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786289732823074-3431-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 15:35:51.890842 6814 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 15:35:51.891125 6814 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 15:35:51.891197 6814 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 15:35:51.895601 6814 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 15:35:51.895700 6814 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 15:35:51.895805 6814 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.3.89.193
I20260809 15:35:51.901113 6814 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.3.89.212:45083
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786289732823074-3431-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786289732823074-3431-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786289732823074-3431-0/krb5kdc/kudu_127.3.89.193.keytab
--principal=kudu/127.3.89.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.3.89.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786289732823074-3431-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.3.89.193
--webserver_port=0
--tserver_master_addrs=127.3.89.254:44261
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.6814
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.3.89.193
--log_dir=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786289732823074-3431-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 09 Aug 2026 14:46:55 UTC on ccycloud.quasar-yswkmf.root.comops.site
build id 4576540
I20260809 15:35:51.902284 6814 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 15:35:51.903409 6814 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 15:35:51.913023 6821 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 15:35:51.913160 6814 server_base.cc:1034] running on GCE node
W20260809 15:35:51.913043 6820 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 15:35:51.913048 6823 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 15:35:51.913725 6814 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 15:35:51.914342 6814 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 15:35:51.915503 6814 hybrid_clock.cc:648] HybridClock initialized: now 1786289751915469 us; error 46 us; skew 500 ppm
Aug 09 15:35:51 dist-test-slave-2w3w krb5kdc[6701](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786289751, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.89.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 15:35:51.918255 6814 init.cc:377] Logged in from keytab as kudu/127.3.89.193@KRBTEST.COM (short username kudu)
I20260809 15:35:51.919368 6814 webserver.cc:460] Webserver started at http://127.3.89.193:40721/ using document root <none> and password file <none>
I20260809 15:35:51.920114 6814 fs_manager.cc:359] Metadata directory not provided
I20260809 15:35:51.920207 6814 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 15:35:51.920424 6814 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 15:35:51.922422 6814 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786289732823074-3431-0/minicluster-data/ts-0/data/instance:
uuid: "0bc34be4998f42aa9b4cc0401d154153"
format_stamp: "Formatted at 2026-08-09 15:35:51 on dist-test-slave-2w3w"
server_key: "d03920c870dfdff32d0d25fa14bbd853"
server_key_iv: "c72dd6a244192674a3585e15ff426752"
server_key_version: "encryptionkey@0"
I20260809 15:35:51.923056 6814 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786289732823074-3431-0/minicluster-data/ts-0/wal/instance:
uuid: "0bc34be4998f42aa9b4cc0401d154153"
format_stamp: "Formatted at 2026-08-09 15:35:51 on dist-test-slave-2w3w"
server_key: "d03920c870dfdff32d0d25fa14bbd853"
server_key_iv: "c72dd6a244192674a3585e15ff426752"
server_key_version: "encryptionkey@0"
I20260809 15:35:51.926649 6814 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.003s sys 0.000s
I20260809 15:35:51.929018 6831 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 15:35:51.930176 6814 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260809 15:35:51.930315 6814 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786289732823074-3431-0/minicluster-data/ts-0/data,/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786289732823074-3431-0/minicluster-data/ts-0/wal
uuid: "0bc34be4998f42aa9b4cc0401d154153"
format_stamp: "Formatted at 2026-08-09 15:35:51 on dist-test-slave-2w3w"
server_key: "d03920c870dfdff32d0d25fa14bbd853"
server_key_iv: "c72dd6a244192674a3585e15ff426752"
server_key_version: "encryptionkey@0"
I20260809 15:35:51.930426 6814 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786289732823074-3431-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786289732823074-3431-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786289732823074-3431-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 15:35:51.940266 6814 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 15:35:51.943347 6814 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 15:35:51.943568 6814 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 15:35:51.944171 6814 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 15:35:51.945012 6814 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 15:35:51.945086 6814 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 15:35:51.945155 6814 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 15:35:51.945205 6814 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 15:35:51.954321 6814 rpc_server.cc:307] RPC server started. Bound to: 127.3.89.193:33067
I20260809 15:35:51.954370 6951 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.3.89.193:33067 every 8 connection(s)
I20260809 15:35:51.955320 6814 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786289732823074-3431-0/minicluster-data/ts-0/data/info.pb
I20260809 15:35:51.963338 3431 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskm314YR/build/debug/bin/kudu as pid 6814
I20260809 15:35:51.963447 3431 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786289732823074-3431-0/minicluster-data/ts-0/wal/instance
I20260809 15:35:51.963908 3431 external_mini_cluster.cc:1428] Setting key fa130ae25af5f5d907270fd03e91f279
Aug 09 15:35:51 dist-test-slave-2w3w krb5kdc[6701](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786289751, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.89.193@KRBTEST.COM for kudu/127.3.89.254@KRBTEST.COM
I20260809 15:35:51.970242 6789 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:35:51.957185 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.89.193:51119 (local address 127.3.89.254:44261)
0809 15:35:51.957353 (+ 168us) server_negotiation.cc:206] Beginning negotiation
0809 15:35:51.957358 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 15:35:51.958155 (+ 797us) server_negotiation.cc:407] Connection header received
0809 15:35:51.959544 (+ 1389us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 15:35:51.959547 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 15:35:51.959597 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 15:35:51.959675 (+ 78us) server_negotiation.cc:226] Negotiated authn=SASL
0809 15:35:51.960814 (+ 1139us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:35:51.961401 (+ 587us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:35:51.962570 (+ 1169us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:35:51.962839 (+ 269us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:35:51.962930 (+ 91us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:35:51.966033 (+ 3103us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 15:35:51.966040 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 15:35:51.966046 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 15:35:51.966079 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0809 15:35:51.967879 (+ 1800us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 15:35:51.968547 (+ 668us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 15:35:51.968549 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 15:35:51.968551 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 15:35:51.968631 (+ 80us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 15:35:51.969075 (+ 444us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 15:35:51.969077 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 15:35:51.969078 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 15:35:51.969260 (+ 182us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 15:35:51.969342 (+ 82us) server_negotiation.cc:1035] Waiting for connection context
0809 15:35:51.970008 (+ 666us) server_negotiation.cc:299] Negotiation successful
0809 15:35:51.970119 (+ 111us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":64}
I20260809 15:35:51.971614 6955 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:35:51.957566 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.89.254:44261 (local address 127.3.89.193:51119)
0809 15:35:51.958001 (+ 435us) negotiation.cc:107] Waiting for socket to connect
0809 15:35:51.958045 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0809 15:35:51.959344 (+ 1299us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 15:35:51.959827 (+ 483us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 15:35:51.959833 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 15:35:51.960175 (+ 342us) client_negotiation.cc:189] Negotiated authn=SASL
0809 15:35:51.960659 (+ 484us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:35:51.960679 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:35:51.961521 (+ 842us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:35:51.961523 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:35:51.962436 (+ 913us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:35:51.962445 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:35:51.963037 (+ 592us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:35:51.963040 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:35:51.963262 (+ 222us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:35:51.964047 (+ 785us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 15:35:51.964063 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0809 15:35:51.965859 (+ 1796us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 15:35:51.968007 (+ 2148us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 15:35:51.968017 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 15:35:51.968036 (+ 19us) client_negotiation.cc:760] calling sasl_client_step()
0809 15:35:51.968434 (+ 398us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 15:35:51.968745 (+ 311us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 15:35:51.968750 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 15:35:51.968753 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0809 15:35:51.968955 (+ 202us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 15:35:51.969377 (+ 422us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 15:35:51.969386 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 15:35:51.969729 (+ 343us) client_negotiation.cc:769] Sending connection context
0809 15:35:51.969953 (+ 224us) client_negotiation.cc:240] Negotiation successful
0809 15:35:51.970346 (+ 393us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":253,"thread_start_us":109,"threads_started":1}
I20260809 15:35:51.973126 6953 heartbeater.cc:347] Connected to a master server at 127.3.89.254:44261
I20260809 15:35:51.973464 6953 heartbeater.cc:464] Registering TS with master...
I20260809 15:35:51.974009 6953 heartbeater.cc:511] Master 127.3.89.254:44261 requested a full tablet report, sending...
WARNING: no policy specified for kudu/127.3.89.194@KRBTEST.COM; defaulting to no policy
I20260809 15:35:51.975951 6748 ts_manager.cc:194] Registered new tserver with Master: 0bc34be4998f42aa9b4cc0401d154153 (127.3.89.193:33067)
I20260809 15:35:51.977305 6748 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.3.89.193@KRBTEST.COM'} at 127.3.89.193:51119
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.3.89.194@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.3.89.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786289732823074-3431-0/krb5kdc/kudu_127.3.89.194.keytab.
Entry for principal kudu/127.3.89.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786289732823074-3431-0/krb5kdc/kudu_127.3.89.194.keytab.
I20260809 15:35:51.995172 3431 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskm314YR/build/debug/bin/kudu
/tmp/dist-test-taskm314YR/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786289732823074-3431-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786289732823074-3431-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786289732823074-3431-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786289732823074-3431-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.3.89.194:0
--local_ip_for_outbound_sockets=127.3.89.194
--webserver_interface=127.3.89.194
--webserver_port=0
--tserver_master_addrs=127.3.89.254:44261
--keytab_file=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786289732823074-3431-0/krb5kdc/kudu_127.3.89.194.keytab
--principal=kudu/127.3.89.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.3.89.212:45083
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786289732823074-3431-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786289732823074-3431-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786289732823074-3431-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 15:35:52.106880 6958 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 15:35:52.107209 6958 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 15:35:52.107254 6958 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 15:35:52.111231 6958 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 15:35:52.111280 6958 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 15:35:52.111336 6958 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.3.89.194
I20260809 15:35:52.116392 6958 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.3.89.212:45083
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786289732823074-3431-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786289732823074-3431-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786289732823074-3431-0/krb5kdc/kudu_127.3.89.194.keytab
--principal=kudu/127.3.89.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.3.89.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786289732823074-3431-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.3.89.194
--webserver_port=0
--tserver_master_addrs=127.3.89.254:44261
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.6958
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.3.89.194
--log_dir=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786289732823074-3431-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 09 Aug 2026 14:46:55 UTC on ccycloud.quasar-yswkmf.root.comops.site
build id 4576540
I20260809 15:35:52.117329 6958 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 15:35:52.118268 6958 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 15:35:52.127866 6966 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 15:35:52.127898 6963 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 15:35:52.128021 6958 server_base.cc:1034] running on GCE node
W20260809 15:35:52.128100 6964 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 15:35:52.128528 6958 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 15:35:52.129159 6958 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 15:35:52.130350 6958 hybrid_clock.cc:648] HybridClock initialized: now 1786289752130338 us; error 36 us; skew 500 ppm
Aug 09 15:35:52 dist-test-slave-2w3w krb5kdc[6701](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786289752, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.89.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 15:35:52.133212 6958 init.cc:377] Logged in from keytab as kudu/127.3.89.194@KRBTEST.COM (short username kudu)
I20260809 15:35:52.134183 6958 webserver.cc:460] Webserver started at http://127.3.89.194:42771/ using document root <none> and password file <none>
I20260809 15:35:52.134835 6958 fs_manager.cc:359] Metadata directory not provided
I20260809 15:35:52.134900 6958 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 15:35:52.135149 6958 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 15:35:52.137274 6958 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786289732823074-3431-0/minicluster-data/ts-1/data/instance:
uuid: "c6d8ab7942d2490d95396f3eaf6dd451"
format_stamp: "Formatted at 2026-08-09 15:35:52 on dist-test-slave-2w3w"
server_key: "00bc21c80404f3b552c32479fbeacf11"
server_key_iv: "c7fe4e2729761635d89776af6482b590"
server_key_version: "encryptionkey@0"
I20260809 15:35:52.137854 6958 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786289732823074-3431-0/minicluster-data/ts-1/wal/instance:
uuid: "c6d8ab7942d2490d95396f3eaf6dd451"
format_stamp: "Formatted at 2026-08-09 15:35:52 on dist-test-slave-2w3w"
server_key: "00bc21c80404f3b552c32479fbeacf11"
server_key_iv: "c7fe4e2729761635d89776af6482b590"
server_key_version: "encryptionkey@0"
I20260809 15:35:52.141754 6958 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260809 15:35:52.144182 6974 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 15:35:52.145371 6958 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260809 15:35:52.145519 6958 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786289732823074-3431-0/minicluster-data/ts-1/data,/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786289732823074-3431-0/minicluster-data/ts-1/wal
uuid: "c6d8ab7942d2490d95396f3eaf6dd451"
format_stamp: "Formatted at 2026-08-09 15:35:52 on dist-test-slave-2w3w"
server_key: "00bc21c80404f3b552c32479fbeacf11"
server_key_iv: "c7fe4e2729761635d89776af6482b590"
server_key_version: "encryptionkey@0"
I20260809 15:35:52.145632 6958 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786289732823074-3431-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786289732823074-3431-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786289732823074-3431-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 15:35:52.159257 6958 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 15:35:52.161959 6958 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 15:35:52.162147 6958 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 15:35:52.162765 6958 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 15:35:52.163596 6958 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 15:35:52.163669 6958 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 15:35:52.163735 6958 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 15:35:52.163789 6958 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 15:35:52.172959 6958 rpc_server.cc:307] RPC server started. Bound to: 127.3.89.194:37583
I20260809 15:35:52.173002 7091 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.3.89.194:37583 every 8 connection(s)
I20260809 15:35:52.173965 6958 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786289732823074-3431-0/minicluster-data/ts-1/data/info.pb
I20260809 15:35:52.182372 3431 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskm314YR/build/debug/bin/kudu as pid 6958
I20260809 15:35:52.182514 3431 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786289732823074-3431-0/minicluster-data/ts-1/wal/instance
I20260809 15:35:52.182819 3431 external_mini_cluster.cc:1428] Setting key 2a960be22e2ed99f78e90e53d1c0e53b
Aug 09 15:35:52 dist-test-slave-2w3w krb5kdc[6701](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786289752, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.89.194@KRBTEST.COM for kudu/127.3.89.254@KRBTEST.COM
I20260809 15:35:52.188544 6789 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:35:52.175990 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.89.194:54203 (local address 127.3.89.254:44261)
0809 15:35:52.176156 (+ 166us) server_negotiation.cc:206] Beginning negotiation
0809 15:35:52.176160 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 15:35:52.177060 (+ 900us) server_negotiation.cc:407] Connection header received
0809 15:35:52.178344 (+ 1284us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 15:35:52.178347 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 15:35:52.178393 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 15:35:52.178471 (+ 78us) server_negotiation.cc:226] Negotiated authn=SASL
0809 15:35:52.179677 (+ 1206us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:35:52.180226 (+ 549us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:35:52.181349 (+ 1123us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:35:52.181587 (+ 238us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:35:52.181676 (+ 89us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:35:52.184651 (+ 2975us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 15:35:52.184653 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 15:35:52.184655 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 15:35:52.184676 (+ 21us) server_negotiation.cc:922] calling sasl_server_start()
0809 15:35:52.186061 (+ 1385us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 15:35:52.186714 (+ 653us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 15:35:52.186719 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 15:35:52.186723 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 15:35:52.186832 (+ 109us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 15:35:52.187236 (+ 404us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 15:35:52.187242 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 15:35:52.187245 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 15:35:52.187505 (+ 260us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 15:35:52.187627 (+ 122us) server_negotiation.cc:1035] Waiting for connection context
0809 15:35:52.188143 (+ 516us) server_negotiation.cc:299] Negotiation successful
0809 15:35:52.188290 (+ 147us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":54}
I20260809 15:35:52.189222 7095 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:35:52.176453 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.89.254:44261 (local address 127.3.89.194:54203)
0809 15:35:52.176891 (+ 438us) negotiation.cc:107] Waiting for socket to connect
0809 15:35:52.176934 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0809 15:35:52.178146 (+ 1212us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 15:35:52.178629 (+ 483us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 15:35:52.178635 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 15:35:52.179052 (+ 417us) client_negotiation.cc:189] Negotiated authn=SASL
0809 15:35:52.179522 (+ 470us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:35:52.179536 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:35:52.180343 (+ 807us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:35:52.180346 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:35:52.181217 (+ 871us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:35:52.181226 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:35:52.181728 (+ 502us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:35:52.181731 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:35:52.181874 (+ 143us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:35:52.182588 (+ 714us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 15:35:52.182616 (+ 28us) client_negotiation.cc:656] calling sasl_client_start()
0809 15:35:52.184470 (+ 1854us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 15:35:52.186179 (+ 1709us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 15:35:52.186188 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 15:35:52.186204 (+ 16us) client_negotiation.cc:760] calling sasl_client_step()
0809 15:35:52.186596 (+ 392us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 15:35:52.186937 (+ 341us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 15:35:52.186939 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 15:35:52.186940 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0809 15:35:52.187135 (+ 195us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 15:35:52.187642 (+ 507us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 15:35:52.187647 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 15:35:52.187880 (+ 233us) client_negotiation.cc:769] Sending connection context
0809 15:35:52.188034 (+ 154us) client_negotiation.cc:240] Negotiation successful
0809 15:35:52.188295 (+ 261us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":258,"thread_start_us":115,"threads_started":1}
I20260809 15:35:52.190546 7092 heartbeater.cc:347] Connected to a master server at 127.3.89.254:44261
I20260809 15:35:52.190793 7092 heartbeater.cc:464] Registering TS with master...
I20260809 15:35:52.191326 7092 heartbeater.cc:511] Master 127.3.89.254:44261 requested a full tablet report, sending...
I20260809 15:35:52.192370 6748 ts_manager.cc:194] Registered new tserver with Master: c6d8ab7942d2490d95396f3eaf6dd451 (127.3.89.194:37583)
I20260809 15:35:52.193192 6748 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.3.89.194@KRBTEST.COM'} at 127.3.89.194:54203
WARNING: no policy specified for kudu/127.3.89.195@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.3.89.195@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.3.89.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786289732823074-3431-0/krb5kdc/kudu_127.3.89.195.keytab.
Entry for principal kudu/127.3.89.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786289732823074-3431-0/krb5kdc/kudu_127.3.89.195.keytab.
I20260809 15:35:52.211534 3431 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskm314YR/build/debug/bin/kudu
/tmp/dist-test-taskm314YR/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786289732823074-3431-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786289732823074-3431-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786289732823074-3431-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786289732823074-3431-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.3.89.195:0
--local_ip_for_outbound_sockets=127.3.89.195
--webserver_interface=127.3.89.195
--webserver_port=0
--tserver_master_addrs=127.3.89.254:44261
--keytab_file=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786289732823074-3431-0/krb5kdc/kudu_127.3.89.195.keytab
--principal=kudu/127.3.89.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.3.89.212:45083
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786289732823074-3431-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786289732823074-3431-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786289732823074-3431-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 15:35:52.321941 7098 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 15:35:52.322228 7098 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 15:35:52.322306 7098 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 15:35:52.326335 7098 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 15:35:52.326406 7098 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 15:35:52.326491 7098 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.3.89.195
I20260809 15:35:52.331725 7098 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.3.89.212:45083
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786289732823074-3431-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786289732823074-3431-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786289732823074-3431-0/krb5kdc/kudu_127.3.89.195.keytab
--principal=kudu/127.3.89.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.3.89.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786289732823074-3431-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.3.89.195
--webserver_port=0
--tserver_master_addrs=127.3.89.254:44261
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.7098
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.3.89.195
--log_dir=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786289732823074-3431-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 09 Aug 2026 14:46:55 UTC on ccycloud.quasar-yswkmf.root.comops.site
build id 4576540
I20260809 15:35:52.332801 7098 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 15:35:52.333969 7098 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 15:35:52.343993 7106 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 15:35:52.343976 7103 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 15:35:52.344133 7098 server_base.cc:1034] running on GCE node
W20260809 15:35:52.344040 7104 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 15:35:52.344686 7098 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 15:35:52.345314 7098 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 15:35:52.346498 7098 hybrid_clock.cc:648] HybridClock initialized: now 1786289752346483 us; error 25 us; skew 500 ppm
Aug 09 15:35:52 dist-test-slave-2w3w krb5kdc[6701](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786289752, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.89.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 15:35:52.349436 7098 init.cc:377] Logged in from keytab as kudu/127.3.89.195@KRBTEST.COM (short username kudu)
I20260809 15:35:52.350234 7098 webserver.cc:460] Webserver started at http://127.3.89.195:42725/ using document root <none> and password file <none>
I20260809 15:35:52.350786 7098 fs_manager.cc:359] Metadata directory not provided
I20260809 15:35:52.350876 7098 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 15:35:52.351132 7098 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 15:35:52.353094 7098 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786289732823074-3431-0/minicluster-data/ts-2/data/instance:
uuid: "7bc7257b6eb3407e9f3a71e3bf903d5b"
format_stamp: "Formatted at 2026-08-09 15:35:52 on dist-test-slave-2w3w"
server_key: "05d5dd153581019b708a186f075ed912"
server_key_iv: "948d8486bbaf46a9cd7857c72c163772"
server_key_version: "encryptionkey@0"
I20260809 15:35:52.353647 7098 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786289732823074-3431-0/minicluster-data/ts-2/wal/instance:
uuid: "7bc7257b6eb3407e9f3a71e3bf903d5b"
format_stamp: "Formatted at 2026-08-09 15:35:52 on dist-test-slave-2w3w"
server_key: "05d5dd153581019b708a186f075ed912"
server_key_iv: "948d8486bbaf46a9cd7857c72c163772"
server_key_version: "encryptionkey@0"
I20260809 15:35:52.357203 7098 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.001s sys 0.004s
I20260809 15:35:52.359438 7113 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 15:35:52.360585 7098 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260809 15:35:52.360716 7098 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786289732823074-3431-0/minicluster-data/ts-2/data,/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786289732823074-3431-0/minicluster-data/ts-2/wal
uuid: "7bc7257b6eb3407e9f3a71e3bf903d5b"
format_stamp: "Formatted at 2026-08-09 15:35:52 on dist-test-slave-2w3w"
server_key: "05d5dd153581019b708a186f075ed912"
server_key_iv: "948d8486bbaf46a9cd7857c72c163772"
server_key_version: "encryptionkey@0"
I20260809 15:35:52.360823 7098 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786289732823074-3431-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786289732823074-3431-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786289732823074-3431-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 15:35:52.386758 7098 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 15:35:52.390107 7098 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 15:35:52.390327 7098 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 15:35:52.390931 7098 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 15:35:52.391772 7098 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 15:35:52.391851 7098 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 15:35:52.391924 7098 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 15:35:52.391969 7098 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 15:35:52.401249 7098 rpc_server.cc:307] RPC server started. Bound to: 127.3.89.195:46877
I20260809 15:35:52.401274 7234 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.3.89.195:46877 every 8 connection(s)
I20260809 15:35:52.402093 7098 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786289732823074-3431-0/minicluster-data/ts-2/data/info.pb
I20260809 15:35:52.408401 3431 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskm314YR/build/debug/bin/kudu as pid 7098
I20260809 15:35:52.408532 3431 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786289732823074-3431-0/minicluster-data/ts-2/wal/instance
I20260809 15:35:52.408828 3431 external_mini_cluster.cc:1428] Setting key 2ffff73f1fab2bb15aa032452d74f338
Aug 09 15:35:52 dist-test-slave-2w3w krb5kdc[6701](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786289752, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.89.195@KRBTEST.COM for kudu/127.3.89.254@KRBTEST.COM
I20260809 15:35:52.415877 6789 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:35:52.403916 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.89.195:49267 (local address 127.3.89.254:44261)
0809 15:35:52.404069 (+ 153us) server_negotiation.cc:206] Beginning negotiation
0809 15:35:52.404072 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0809 15:35:52.404875 (+ 803us) server_negotiation.cc:407] Connection header received
0809 15:35:52.406293 (+ 1418us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 15:35:52.406297 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 15:35:52.406356 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 15:35:52.406469 (+ 113us) server_negotiation.cc:226] Negotiated authn=SASL
0809 15:35:52.407752 (+ 1283us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:35:52.408301 (+ 549us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:35:52.409701 (+ 1400us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:35:52.409925 (+ 224us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:35:52.410031 (+ 106us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:35:52.412466 (+ 2435us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 15:35:52.412471 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 15:35:52.412479 (+ 8us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 15:35:52.412502 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0809 15:35:52.413717 (+ 1215us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 15:35:52.414226 (+ 509us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 15:35:52.414230 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 15:35:52.414232 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 15:35:52.414325 (+ 93us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 15:35:52.414683 (+ 358us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 15:35:52.414687 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 15:35:52.414688 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 15:35:52.414905 (+ 217us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 15:35:52.415051 (+ 146us) server_negotiation.cc:1035] Waiting for connection context
0809 15:35:52.415563 (+ 512us) server_negotiation.cc:299] Negotiation successful
0809 15:35:52.415710 (+ 147us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":57}
I20260809 15:35:52.416729 7237 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:35:52.404242 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.89.254:44261 (local address 127.3.89.195:49267)
0809 15:35:52.404706 (+ 464us) negotiation.cc:107] Waiting for socket to connect
0809 15:35:52.404758 (+ 52us) client_negotiation.cc:174] Beginning negotiation
0809 15:35:52.406075 (+ 1317us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 15:35:52.406638 (+ 563us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 15:35:52.406645 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 15:35:52.407056 (+ 411us) client_negotiation.cc:189] Negotiated authn=SASL
0809 15:35:52.407608 (+ 552us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:35:52.407621 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:35:52.408428 (+ 807us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:35:52.408434 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:35:52.409572 (+ 1138us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:35:52.409581 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:35:52.410077 (+ 496us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:35:52.410081 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:35:52.410249 (+ 168us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:35:52.410839 (+ 590us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 15:35:52.410854 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0809 15:35:52.412307 (+ 1453us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 15:35:52.413839 (+ 1532us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 15:35:52.413845 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 15:35:52.413859 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0809 15:35:52.414118 (+ 259us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 15:35:52.414429 (+ 311us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 15:35:52.414432 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 15:35:52.414434 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 15:35:52.414576 (+ 142us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 15:35:52.415046 (+ 470us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 15:35:52.415052 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 15:35:52.415321 (+ 269us) client_negotiation.cc:769] Sending connection context
0809 15:35:52.415521 (+ 200us) client_negotiation.cc:240] Negotiation successful
0809 15:35:52.415875 (+ 354us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":291,"thread_start_us":110,"threads_started":1}
I20260809 15:35:52.417906 7235 heartbeater.cc:347] Connected to a master server at 127.3.89.254:44261
I20260809 15:35:52.418166 7235 heartbeater.cc:464] Registering TS with master...
I20260809 15:35:52.418682 7235 heartbeater.cc:511] Master 127.3.89.254:44261 requested a full tablet report, sending...
I20260809 15:35:52.419876 6748 ts_manager.cc:194] Registered new tserver with Master: 7bc7257b6eb3407e9f3a71e3bf903d5b (127.3.89.195:46877)
I20260809 15:35:52.420485 6748 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.3.89.195@KRBTEST.COM'} at 127.3.89.195:49267
I20260809 15:35:52.422935 3431 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Aug 09 15:35:52 dist-test-slave-2w3w krb5kdc[6701](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786289752, etypes {rep=17 tkt=17 ses=17}, joe-interloper@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for joe-interloper@KRBTEST.COM:
Aug 09 15:35:52 dist-test-slave-2w3w krb5kdc[6701](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786289752, etypes {rep=17 tkt=17 ses=17}, joe-interloper@KRBTEST.COM for kudu/127.3.89.254@KRBTEST.COM
I20260809 15:35:52.459115 6789 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:35:52.446891 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33182 (local address 127.3.89.254:44261)
0809 15:35:52.447066 (+ 175us) server_negotiation.cc:206] Beginning negotiation
0809 15:35:52.447073 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 15:35:52.447867 (+ 794us) server_negotiation.cc:407] Connection header received
0809 15:35:52.448007 (+ 140us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 15:35:52.448009 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 15:35:52.448058 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 15:35:52.448168 (+ 110us) server_negotiation.cc:226] Negotiated authn=SASL
0809 15:35:52.449142 (+ 974us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:35:52.450020 (+ 878us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:35:52.451470 (+ 1450us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:35:52.451884 (+ 414us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:35:52.451989 (+ 105us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:35:52.455254 (+ 3265us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 15:35:52.455257 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 15:35:52.455260 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 15:35:52.455299 (+ 39us) server_negotiation.cc:922] calling sasl_server_start()
0809 15:35:52.457138 (+ 1839us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 15:35:52.457591 (+ 453us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 15:35:52.457595 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 15:35:52.457597 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 15:35:52.457684 (+ 87us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 15:35:52.457985 (+ 301us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 15:35:52.457992 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 15:35:52.457994 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 15:35:52.458319 (+ 325us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 15:35:52.458441 (+ 122us) server_negotiation.cc:1035] Waiting for connection context
0809 15:35:52.458795 (+ 354us) server_negotiation.cc:299] Negotiation successful
0809 15:35:52.458932 (+ 137us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":81}
W20260809 15:35:52.459642 6748 server_base.cc:1116] Unauthorized access attempt to method kudu.master.MasterService.ConnectToMaster from {username='joe-interloper', principal='joe-interloper@KRBTEST.COM'} at 127.0.0.1:33182
I20260809 15:35:52.460562 3431 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskm314YR/build/debug/bin/kudu with pid 6814
I20260809 15:35:52.477926 3431 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskm314YR/build/debug/bin/kudu with pid 6958
I20260809 15:35:52.487440 3431 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskm314YR/build/debug/bin/kudu with pid 7098
I20260809 15:35:52.493310 3431 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskm314YR/build/debug/bin/kudu with pid 6715
2026-08-09T15:35:52Z chronyd exiting
[ OK ] SecurityITest.TestUnauthorizedClientKerberosCredentials (2233 ms)
[ RUN ] SecurityITest.TestAuthorizedSuperuser
Loading random data
Initializing database '/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786289732823074-3431-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 15:35:52 dist-test-slave-2w3w krb5kdc[7247](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 15:35:52 dist-test-slave-2w3w krb5kdc[7247](info): set up 2 sockets
Aug 09 15:35:52 dist-test-slave-2w3w krb5kdc[7247](info): commencing operation
krb5kdc: starting...
W20260809 15:35:53.580546 3431 mini_kdc.cc:121] Time spent starting KDC: real 1.063s user 0.001s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786289732823074-3431-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786289732823074-3431-0/krb5kdc/test-user.keytab.
Aug 09 15:35:53 dist-test-slave-2w3w krb5kdc[7247](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786289753, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T15:35:53Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T15:35:53Z Disabled control of system clock
WARNING: no policy specified for kudu/127.3.89.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.3.89.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.3.89.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786289732823074-3431-0/krb5kdc/kudu_127.3.89.254.keytab.
Entry for principal kudu/127.3.89.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786289732823074-3431-0/krb5kdc/kudu_127.3.89.254.keytab.
I20260809 15:35:53.703812 3431 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskm314YR/build/debug/bin/kudu
/tmp/dist-test-taskm314YR/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786289732823074-3431-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786289732823074-3431-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786289732823074-3431-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786289732823074-3431-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.3.89.254:33917
--webserver_interface=127.3.89.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786289732823074-3431-0/krb5kdc/kudu_127.3.89.254.keytab
--principal=kudu/127.3.89.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.3.89.212:46551
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.3.89.254:33917
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786289732823074-3431-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786289732823074-3431-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786289732823074-3431-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 15:35:53.813108 7261 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 15:35:53.813344 7261 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 15:35:53.813385 7261 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 15:35:53.817301 7261 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 15:35:53.817351 7261 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 15:35:53.817370 7261 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 15:35:53.817386 7261 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 15:35:53.817401 7261 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 15:35:53.822811 7261 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.3.89.212:46551
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786289732823074-3431-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786289732823074-3431-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.3.89.254:33917
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786289732823074-3431-0/krb5kdc/kudu_127.3.89.254.keytab
--principal=kudu/127.3.89.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.3.89.254:33917
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786289732823074-3431-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.3.89.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.7261
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786289732823074-3431-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 09 Aug 2026 14:46:55 UTC on ccycloud.quasar-yswkmf.root.comops.site
build id 4576540
I20260809 15:35:53.823822 7261 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 15:35:53.824743 7261 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 15:35:53.833112 7269 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 15:35:53.833123 7267 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 15:35:53.833123 7266 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 15:35:53.833262 7261 server_base.cc:1034] running on GCE node
I20260809 15:35:53.833849 7261 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 15:35:53.834826 7261 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 15:35:53.836016 7261 hybrid_clock.cc:648] HybridClock initialized: now 1786289753836006 us; error 29 us; skew 500 ppm
Aug 09 15:35:53 dist-test-slave-2w3w krb5kdc[7247](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786289753, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.89.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 15:35:53.844309 7261 init.cc:377] Logged in from keytab as kudu/127.3.89.254@KRBTEST.COM (short username kudu)
I20260809 15:35:53.845216 7261 webserver.cc:460] Webserver started at http://127.3.89.254:41371/ using document root <none> and password file <none>
I20260809 15:35:53.845770 7261 fs_manager.cc:359] Metadata directory not provided
I20260809 15:35:53.845863 7261 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 15:35:53.846097 7261 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 15:35:53.848162 7261 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786289732823074-3431-0/minicluster-data/master-0/data/instance:
uuid: "2ddb4a1aae99491abe69922ea25720ac"
format_stamp: "Formatted at 2026-08-09 15:35:53 on dist-test-slave-2w3w"
server_key: "4c153e66019a6c338d20fedb9de5983b"
server_key_iv: "b7ac97163a2e103db093a2523f3540de"
server_key_version: "encryptionkey@0"
I20260809 15:35:53.848719 7261 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786289732823074-3431-0/minicluster-data/master-0/wal/instance:
uuid: "2ddb4a1aae99491abe69922ea25720ac"
format_stamp: "Formatted at 2026-08-09 15:35:53 on dist-test-slave-2w3w"
server_key: "4c153e66019a6c338d20fedb9de5983b"
server_key_iv: "b7ac97163a2e103db093a2523f3540de"
server_key_version: "encryptionkey@0"
I20260809 15:35:53.852313 7261 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.004s sys 0.000s
I20260809 15:35:53.854549 7278 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 15:35:53.855789 7261 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260809 15:35:53.855932 7261 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786289732823074-3431-0/minicluster-data/master-0/data,/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786289732823074-3431-0/minicluster-data/master-0/wal
uuid: "2ddb4a1aae99491abe69922ea25720ac"
format_stamp: "Formatted at 2026-08-09 15:35:53 on dist-test-slave-2w3w"
server_key: "4c153e66019a6c338d20fedb9de5983b"
server_key_iv: "b7ac97163a2e103db093a2523f3540de"
server_key_version: "encryptionkey@0"
I20260809 15:35:53.856043 7261 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786289732823074-3431-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786289732823074-3431-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786289732823074-3431-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 15:35:53.884269 7261 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 15:35:53.887280 7261 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 15:35:53.887508 7261 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 15:35:53.894680 7261 rpc_server.cc:307] RPC server started. Bound to: 127.3.89.254:33917
I20260809 15:35:53.894714 7334 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.3.89.254:33917 every 8 connection(s)
I20260809 15:35:53.895665 7261 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786289732823074-3431-0/minicluster-data/master-0/data/info.pb
I20260809 15:35:53.898267 7335 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 15:35:53.900681 3431 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskm314YR/build/debug/bin/kudu as pid 7261
I20260809 15:35:53.900815 3431 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786289732823074-3431-0/minicluster-data/master-0/wal/instance
I20260809 15:35:53.901124 3431 external_mini_cluster.cc:1428] Setting key 663f144c2bb04619a70ad4f1b7cfb211
I20260809 15:35:53.904361 7335 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2ddb4a1aae99491abe69922ea25720ac: Bootstrap starting.
I20260809 15:35:53.906455 7335 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 2ddb4a1aae99491abe69922ea25720ac: Neither blocks nor log segments found. Creating new log.
I20260809 15:35:53.907250 7335 log.cc:824] T 00000000000000000000000000000000 P 2ddb4a1aae99491abe69922ea25720ac: Log is configured to *not* fsync() on all Append() calls
I20260809 15:35:53.909591 7335 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2ddb4a1aae99491abe69922ea25720ac: No bootstrap required, opened a new log
Aug 09 15:35:53 dist-test-slave-2w3w krb5kdc[7247](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786289753, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.3.89.254@KRBTEST.COM
I20260809 15:35:53.912982 7335 raft_consensus.cc:348] T 00000000000000000000000000000000 P 2ddb4a1aae99491abe69922ea25720ac [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2ddb4a1aae99491abe69922ea25720ac" member_type: VOTER last_known_addr { host: "127.3.89.254" port: 33917 } }
I20260809 15:35:53.913229 7335 raft_consensus.cc:374] T 00000000000000000000000000000000 P 2ddb4a1aae99491abe69922ea25720ac [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 15:35:53.913295 7335 raft_consensus.cc:729] T 00000000000000000000000000000000 P 2ddb4a1aae99491abe69922ea25720ac [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2ddb4a1aae99491abe69922ea25720ac, State: Initialized, Role: FOLLOWER
I20260809 15:35:53.913817 7335 consensus_queue.cc:260] T 00000000000000000000000000000000 P 2ddb4a1aae99491abe69922ea25720ac [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2ddb4a1aae99491abe69922ea25720ac" member_type: VOTER last_known_addr { host: "127.3.89.254" port: 33917 } }
I20260809 15:35:53.914016 7335 raft_consensus.cc:388] T 00000000000000000000000000000000 P 2ddb4a1aae99491abe69922ea25720ac [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 15:35:53.914088 7335 raft_consensus.cc:482] T 00000000000000000000000000000000 P 2ddb4a1aae99491abe69922ea25720ac [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 15:35:53.914208 7335 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 2ddb4a1aae99491abe69922ea25720ac [term 0 FOLLOWER]: Advancing to term 1
I20260809 15:35:53.915449 7335 raft_consensus.cc:504] T 00000000000000000000000000000000 P 2ddb4a1aae99491abe69922ea25720ac [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2ddb4a1aae99491abe69922ea25720ac" member_type: VOTER last_known_addr { host: "127.3.89.254" port: 33917 } }
I20260809 15:35:53.915815 7335 leader_election.cc:302] T 00000000000000000000000000000000 P 2ddb4a1aae99491abe69922ea25720ac [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 2ddb4a1aae99491abe69922ea25720ac; no voters:
I20260809 15:35:53.916160 7335 leader_election.cc:288] T 00000000000000000000000000000000 P 2ddb4a1aae99491abe69922ea25720ac [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 15:35:53.916308 7340 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 2ddb4a1aae99491abe69922ea25720ac [term 1 FOLLOWER]: Leader election won for term 1
I20260809 15:35:53.916550 7338 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:35:53.902616 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56104 (local address 127.3.89.254:33917)
0809 15:35:53.903079 (+ 463us) server_negotiation.cc:206] Beginning negotiation
0809 15:35:53.903094 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0809 15:35:53.903129 (+ 35us) server_negotiation.cc:407] Connection header received
0809 15:35:53.903766 (+ 637us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 15:35:53.903784 (+ 18us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 15:35:53.904095 (+ 311us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 15:35:53.904431 (+ 336us) server_negotiation.cc:226] Negotiated authn=SASL
0809 15:35:53.905249 (+ 818us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:35:53.906251 (+ 1002us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:35:53.907209 (+ 958us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:35:53.907828 (+ 619us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:35:53.908004 (+ 176us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:35:53.910785 (+ 2781us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 15:35:53.910790 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 15:35:53.910802 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 15:35:53.910827 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0809 15:35:53.913155 (+ 2328us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 15:35:53.913686 (+ 531us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 15:35:53.913692 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 15:35:53.913697 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0809 15:35:53.913840 (+ 143us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 15:35:53.914166 (+ 326us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 15:35:53.914170 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 15:35:53.914190 (+ 20us) server_negotiation.cc:967] calling sasl_server_step()
0809 15:35:53.914738 (+ 548us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 15:35:53.914952 (+ 214us) server_negotiation.cc:1035] Waiting for connection context
0809 15:35:53.915236 (+ 284us) server_negotiation.cc:299] Negotiation successful
0809 15:35:53.915546 (+ 310us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":275,"thread_start_us":126,"threads_started":1}
I20260809 15:35:53.916580 7340 raft_consensus.cc:686] T 00000000000000000000000000000000 P 2ddb4a1aae99491abe69922ea25720ac [term 1 LEADER]: Becoming Leader. State: Replica: 2ddb4a1aae99491abe69922ea25720ac, State: Running, Role: LEADER
I20260809 15:35:53.916880 7340 consensus_queue.cc:237] T 00000000000000000000000000000000 P 2ddb4a1aae99491abe69922ea25720ac [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2ddb4a1aae99491abe69922ea25720ac" member_type: VOTER last_known_addr { host: "127.3.89.254" port: 33917 } }
I20260809 15:35:53.917359 7335 sys_catalog.cc:564] T 00000000000000000000000000000000 P 2ddb4a1aae99491abe69922ea25720ac [sys.catalog]: configured and running, proceeding with master startup.
I20260809 15:35:53.918097 7340 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2ddb4a1aae99491abe69922ea25720ac [sys.catalog]: SysCatalogTable state changed. Reason: New leader 2ddb4a1aae99491abe69922ea25720ac. Latest consensus state: current_term: 1 leader_uuid: "2ddb4a1aae99491abe69922ea25720ac" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2ddb4a1aae99491abe69922ea25720ac" member_type: VOTER last_known_addr { host: "127.3.89.254" port: 33917 } } }
I20260809 15:35:53.918203 7340 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2ddb4a1aae99491abe69922ea25720ac [sys.catalog]: This master's current role is: LEADER
I20260809 15:35:53.919271 7341 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2ddb4a1aae99491abe69922ea25720ac [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "2ddb4a1aae99491abe69922ea25720ac" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2ddb4a1aae99491abe69922ea25720ac" member_type: VOTER last_known_addr { host: "127.3.89.254" port: 33917 } } }
I20260809 15:35:53.919441 7341 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2ddb4a1aae99491abe69922ea25720ac [sys.catalog]: This master's current role is: LEADER
W20260809 15:35:53.922148 7357 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 2ddb4a1aae99491abe69922ea25720ac: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260809 15:35:53.922214 7357 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260809 15:35:53.922271 7349 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 15:35:53.923022 7349 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 15:35:53.928170 7349 catalog_manager.cc:1347] Generated new cluster ID: 3da04ade8e9040feaefed552134ccb3e
I20260809 15:35:53.928232 7349 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 15:35:53.942324 7349 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 15:35:53.943325 7349 catalog_manager.cc:1504] Loading token signing keys...
I20260809 15:35:53.950604 7349 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 2ddb4a1aae99491abe69922ea25720ac: Generated new TSK 0
I20260809 15:35:53.951359 7349 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.3.89.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.3.89.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.3.89.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786289732823074-3431-0/krb5kdc/kudu_127.3.89.193.keytab.
Entry for principal kudu/127.3.89.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786289732823074-3431-0/krb5kdc/kudu_127.3.89.193.keytab.
I20260809 15:35:53.984017 3431 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskm314YR/build/debug/bin/kudu
/tmp/dist-test-taskm314YR/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786289732823074-3431-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786289732823074-3431-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786289732823074-3431-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786289732823074-3431-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.3.89.193:0
--local_ip_for_outbound_sockets=127.3.89.193
--webserver_interface=127.3.89.193
--webserver_port=0
--tserver_master_addrs=127.3.89.254:33917
--keytab_file=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786289732823074-3431-0/krb5kdc/kudu_127.3.89.193.keytab
--principal=kudu/127.3.89.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.3.89.212:46551
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786289732823074-3431-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786289732823074-3431-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786289732823074-3431-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 15:35:54.104604 7363 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 15:35:54.105032 7363 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 15:35:54.105175 7363 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 15:35:54.109311 7363 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 15:35:54.109395 7363 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 15:35:54.109479 7363 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.3.89.193
I20260809 15:35:54.114797 7363 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.3.89.212:46551
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786289732823074-3431-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786289732823074-3431-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786289732823074-3431-0/krb5kdc/kudu_127.3.89.193.keytab
--principal=kudu/127.3.89.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.3.89.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786289732823074-3431-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.3.89.193
--webserver_port=0
--tserver_master_addrs=127.3.89.254:33917
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.7363
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.3.89.193
--log_dir=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786289732823074-3431-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 09 Aug 2026 14:46:55 UTC on ccycloud.quasar-yswkmf.root.comops.site
build id 4576540
I20260809 15:35:54.115835 7363 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 15:35:54.116835 7363 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 15:35:54.126302 7369 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 15:35:54.126310 7372 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 15:35:54.126310 7368 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 15:35:54.126727 7363 server_base.cc:1034] running on GCE node
I20260809 15:35:54.127202 7363 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 15:35:54.127796 7363 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 15:35:54.128963 7363 hybrid_clock.cc:648] HybridClock initialized: now 1786289754128956 us; error 45 us; skew 500 ppm
Aug 09 15:35:54 dist-test-slave-2w3w krb5kdc[7247](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786289754, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.89.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 15:35:54.131793 7363 init.cc:377] Logged in from keytab as kudu/127.3.89.193@KRBTEST.COM (short username kudu)
I20260809 15:35:54.132560 7363 webserver.cc:460] Webserver started at http://127.3.89.193:35075/ using document root <none> and password file <none>
I20260809 15:35:54.133051 7363 fs_manager.cc:359] Metadata directory not provided
I20260809 15:35:54.133106 7363 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 15:35:54.133265 7363 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 15:35:54.135268 7363 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786289732823074-3431-0/minicluster-data/ts-0/data/instance:
uuid: "3270f17cd9ae49a6b0831581914b758c"
format_stamp: "Formatted at 2026-08-09 15:35:54 on dist-test-slave-2w3w"
server_key: "1b1de8a1e6ea5e878846603051881cf1"
server_key_iv: "ac3fa1b2885a6449b87170a57718670b"
server_key_version: "encryptionkey@0"
I20260809 15:35:54.135798 7363 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786289732823074-3431-0/minicluster-data/ts-0/wal/instance:
uuid: "3270f17cd9ae49a6b0831581914b758c"
format_stamp: "Formatted at 2026-08-09 15:35:54 on dist-test-slave-2w3w"
server_key: "1b1de8a1e6ea5e878846603051881cf1"
server_key_iv: "ac3fa1b2885a6449b87170a57718670b"
server_key_version: "encryptionkey@0"
I20260809 15:35:54.139319 7363 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.001s sys 0.003s
I20260809 15:35:54.141527 7379 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 15:35:54.142656 7363 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260809 15:35:54.142773 7363 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786289732823074-3431-0/minicluster-data/ts-0/data,/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786289732823074-3431-0/minicluster-data/ts-0/wal
uuid: "3270f17cd9ae49a6b0831581914b758c"
format_stamp: "Formatted at 2026-08-09 15:35:54 on dist-test-slave-2w3w"
server_key: "1b1de8a1e6ea5e878846603051881cf1"
server_key_iv: "ac3fa1b2885a6449b87170a57718670b"
server_key_version: "encryptionkey@0"
I20260809 15:35:54.142874 7363 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786289732823074-3431-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786289732823074-3431-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786289732823074-3431-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 15:35:54.164026 7363 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 15:35:54.167133 7363 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 15:35:54.167273 7363 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 15:35:54.167876 7363 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 15:35:54.168648 7363 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 15:35:54.168694 7363 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 15:35:54.168759 7363 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 15:35:54.168789 7363 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 15:35:54.177242 7363 rpc_server.cc:307] RPC server started. Bound to: 127.3.89.193:46755
I20260809 15:35:54.177351 7499 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.3.89.193:46755 every 8 connection(s)
I20260809 15:35:54.178130 7363 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786289732823074-3431-0/minicluster-data/ts-0/data/info.pb
I20260809 15:35:54.181023 3431 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskm314YR/build/debug/bin/kudu as pid 7363
I20260809 15:35:54.181164 3431 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786289732823074-3431-0/minicluster-data/ts-0/wal/instance
I20260809 15:35:54.181453 3431 external_mini_cluster.cc:1428] Setting key 3137c28bccc074ada26c4a1a7ba236db
Aug 09 15:35:54 dist-test-slave-2w3w krb5kdc[7247](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786289754, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.89.193@KRBTEST.COM for kudu/127.3.89.254@KRBTEST.COM
WARNING: no policy specified for kudu/127.3.89.194@KRBTEST.COM; defaulting to no policy
I20260809 15:35:54.192767 7338 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:35:54.179910 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.89.193:47115 (local address 127.3.89.254:33917)
0809 15:35:54.180054 (+ 144us) server_negotiation.cc:206] Beginning negotiation
0809 15:35:54.180058 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 15:35:54.180841 (+ 783us) server_negotiation.cc:407] Connection header received
0809 15:35:54.182340 (+ 1499us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 15:35:54.182343 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 15:35:54.182399 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 15:35:54.182486 (+ 87us) server_negotiation.cc:226] Negotiated authn=SASL
0809 15:35:54.183684 (+ 1198us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:35:54.184227 (+ 543us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:35:54.185932 (+ 1705us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:35:54.186158 (+ 226us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:35:54.186242 (+ 84us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:35:54.189115 (+ 2873us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 15:35:54.189118 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 15:35:54.189120 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 15:35:54.189141 (+ 21us) server_negotiation.cc:922] calling sasl_server_start()
0809 15:35:54.190624 (+ 1483us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 15:35:54.191223 (+ 599us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 15:35:54.191227 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 15:35:54.191229 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 15:35:54.191316 (+ 87us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 15:35:54.191709 (+ 393us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 15:35:54.191713 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 15:35:54.191714 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 15:35:54.191890 (+ 176us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 15:35:54.191995 (+ 105us) server_negotiation.cc:1035] Waiting for connection context
0809 15:35:54.192466 (+ 471us) server_negotiation.cc:299] Negotiation successful
0809 15:35:54.192594 (+ 128us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":60}
I20260809 15:35:54.193584 7502 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:35:54.180238 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.89.254:33917 (local address 127.3.89.193:47115)
0809 15:35:54.180681 (+ 443us) negotiation.cc:107] Waiting for socket to connect
0809 15:35:54.180722 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0809 15:35:54.182137 (+ 1415us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 15:35:54.182663 (+ 526us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 15:35:54.182670 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 15:35:54.183079 (+ 409us) client_negotiation.cc:189] Negotiated authn=SASL
0809 15:35:54.183538 (+ 459us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:35:54.183552 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:35:54.184358 (+ 806us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:35:54.184367 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:35:54.185801 (+ 1434us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:35:54.185814 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:35:54.186316 (+ 502us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:35:54.186323 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:35:54.186565 (+ 242us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:35:54.187217 (+ 652us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 15:35:54.187246 (+ 29us) client_negotiation.cc:656] calling sasl_client_start()
0809 15:35:54.188942 (+ 1696us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 15:35:54.190744 (+ 1802us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 15:35:54.190750 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 15:35:54.190762 (+ 12us) client_negotiation.cc:760] calling sasl_client_step()
0809 15:35:54.191121 (+ 359us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 15:35:54.191415 (+ 294us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 15:35:54.191418 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 15:35:54.191420 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 15:35:54.191607 (+ 187us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 15:35:54.191992 (+ 385us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 15:35:54.191998 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 15:35:54.192234 (+ 236us) client_negotiation.cc:769] Sending connection context
0809 15:35:54.192406 (+ 172us) client_negotiation.cc:240] Negotiation successful
0809 15:35:54.192738 (+ 332us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":274,"thread_start_us":113,"threads_started":1}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.3.89.194@KRBTEST.COM" created.
I20260809 15:35:54.194837 7500 heartbeater.cc:347] Connected to a master server at 127.3.89.254:33917
I20260809 15:35:54.195114 7500 heartbeater.cc:464] Registering TS with master...
I20260809 15:35:54.195585 7500 heartbeater.cc:511] Master 127.3.89.254:33917 requested a full tablet report, sending...
I20260809 15:35:54.197057 7295 ts_manager.cc:194] Registered new tserver with Master: 3270f17cd9ae49a6b0831581914b758c (127.3.89.193:46755)
I20260809 15:35:54.198141 7295 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.3.89.193@KRBTEST.COM'} at 127.3.89.193:47115
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.3.89.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786289732823074-3431-0/krb5kdc/kudu_127.3.89.194.keytab.
Entry for principal kudu/127.3.89.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786289732823074-3431-0/krb5kdc/kudu_127.3.89.194.keytab.
I20260809 15:35:54.210552 3431 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskm314YR/build/debug/bin/kudu
/tmp/dist-test-taskm314YR/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786289732823074-3431-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786289732823074-3431-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786289732823074-3431-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786289732823074-3431-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.3.89.194:0
--local_ip_for_outbound_sockets=127.3.89.194
--webserver_interface=127.3.89.194
--webserver_port=0
--tserver_master_addrs=127.3.89.254:33917
--keytab_file=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786289732823074-3431-0/krb5kdc/kudu_127.3.89.194.keytab
--principal=kudu/127.3.89.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.3.89.212:46551
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786289732823074-3431-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786289732823074-3431-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786289732823074-3431-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 15:35:54.318501 7505 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 15:35:54.318775 7505 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 15:35:54.318846 7505 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 15:35:54.322940 7505 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 15:35:54.323040 7505 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 15:35:54.323132 7505 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.3.89.194
I20260809 15:35:54.328219 7505 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.3.89.212:46551
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786289732823074-3431-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786289732823074-3431-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786289732823074-3431-0/krb5kdc/kudu_127.3.89.194.keytab
--principal=kudu/127.3.89.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.3.89.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786289732823074-3431-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.3.89.194
--webserver_port=0
--tserver_master_addrs=127.3.89.254:33917
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.7505
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.3.89.194
--log_dir=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786289732823074-3431-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 09 Aug 2026 14:46:55 UTC on ccycloud.quasar-yswkmf.root.comops.site
build id 4576540
I20260809 15:35:54.329263 7505 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 15:35:54.330241 7505 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 15:35:54.339723 7512 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 15:35:54.339701 7511 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 15:35:54.339689 7514 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 15:35:54.339944 7505 server_base.cc:1034] running on GCE node
I20260809 15:35:54.340342 7505 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 15:35:54.340909 7505 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 15:35:54.342051 7505 hybrid_clock.cc:648] HybridClock initialized: now 1786289754342018 us; error 43 us; skew 500 ppm
Aug 09 15:35:54 dist-test-slave-2w3w krb5kdc[7247](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786289754, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.89.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 15:35:54.344729 7505 init.cc:377] Logged in from keytab as kudu/127.3.89.194@KRBTEST.COM (short username kudu)
I20260809 15:35:54.345626 7505 webserver.cc:460] Webserver started at http://127.3.89.194:34809/ using document root <none> and password file <none>
I20260809 15:35:54.346217 7505 fs_manager.cc:359] Metadata directory not provided
I20260809 15:35:54.346336 7505 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 15:35:54.346611 7505 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 15:35:54.348707 7505 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786289732823074-3431-0/minicluster-data/ts-1/data/instance:
uuid: "75bc8448336b42fca99c65d4b766eeab"
format_stamp: "Formatted at 2026-08-09 15:35:54 on dist-test-slave-2w3w"
server_key: "2d9ec1d8ca1d803119816ef3233f0fef"
server_key_iv: "2b3b14c5efae476ff5390460a6467c7d"
server_key_version: "encryptionkey@0"
I20260809 15:35:54.349267 7505 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786289732823074-3431-0/minicluster-data/ts-1/wal/instance:
uuid: "75bc8448336b42fca99c65d4b766eeab"
format_stamp: "Formatted at 2026-08-09 15:35:54 on dist-test-slave-2w3w"
server_key: "2d9ec1d8ca1d803119816ef3233f0fef"
server_key_iv: "2b3b14c5efae476ff5390460a6467c7d"
server_key_version: "encryptionkey@0"
I20260809 15:35:54.352870 7505 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.005s sys 0.000s
I20260809 15:35:54.355207 7521 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 15:35:54.356364 7505 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260809 15:35:54.356492 7505 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786289732823074-3431-0/minicluster-data/ts-1/data,/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786289732823074-3431-0/minicluster-data/ts-1/wal
uuid: "75bc8448336b42fca99c65d4b766eeab"
format_stamp: "Formatted at 2026-08-09 15:35:54 on dist-test-slave-2w3w"
server_key: "2d9ec1d8ca1d803119816ef3233f0fef"
server_key_iv: "2b3b14c5efae476ff5390460a6467c7d"
server_key_version: "encryptionkey@0"
I20260809 15:35:54.356596 7505 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786289732823074-3431-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786289732823074-3431-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786289732823074-3431-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 15:35:54.374862 7505 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 15:35:54.377837 7505 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 15:35:54.378034 7505 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 15:35:54.378610 7505 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 15:35:54.379482 7505 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 15:35:54.379551 7505 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 15:35:54.379619 7505 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 15:35:54.379667 7505 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 15:35:54.388306 7505 rpc_server.cc:307] RPC server started. Bound to: 127.3.89.194:44065
I20260809 15:35:54.388340 7639 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.3.89.194:44065 every 8 connection(s)
I20260809 15:35:54.389168 7505 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786289732823074-3431-0/minicluster-data/ts-1/data/info.pb
I20260809 15:35:54.396783 3431 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskm314YR/build/debug/bin/kudu as pid 7505
I20260809 15:35:54.396924 3431 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786289732823074-3431-0/minicluster-data/ts-1/wal/instance
I20260809 15:35:54.397258 3431 external_mini_cluster.cc:1428] Setting key 07b4ebf2e037aa1b33ab44d9091525c5
Aug 09 15:35:54 dist-test-slave-2w3w krb5kdc[7247](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786289754, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.89.194@KRBTEST.COM for kudu/127.3.89.254@KRBTEST.COM
I20260809 15:35:54.403654 7338 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:35:54.391030 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.89.194:41277 (local address 127.3.89.254:33917)
0809 15:35:54.391173 (+ 143us) server_negotiation.cc:206] Beginning negotiation
0809 15:35:54.391176 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0809 15:35:54.391951 (+ 775us) server_negotiation.cc:407] Connection header received
0809 15:35:54.393373 (+ 1422us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 15:35:54.393377 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 15:35:54.393434 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 15:35:54.393541 (+ 107us) server_negotiation.cc:226] Negotiated authn=SASL
0809 15:35:54.394669 (+ 1128us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:35:54.395283 (+ 614us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:35:54.396457 (+ 1174us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:35:54.396837 (+ 380us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:35:54.396965 (+ 128us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:35:54.399606 (+ 2641us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 15:35:54.399612 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 15:35:54.399618 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 15:35:54.399652 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0809 15:35:54.401270 (+ 1618us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 15:35:54.401917 (+ 647us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 15:35:54.401920 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 15:35:54.401921 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 15:35:54.401997 (+ 76us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 15:35:54.402397 (+ 400us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 15:35:54.402402 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 15:35:54.402407 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0809 15:35:54.402652 (+ 245us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 15:35:54.402769 (+ 117us) server_negotiation.cc:1035] Waiting for connection context
0809 15:35:54.403267 (+ 498us) server_negotiation.cc:299] Negotiation successful
0809 15:35:54.403406 (+ 139us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":58}
I20260809 15:35:54.404274 7642 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:35:54.391361 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.89.254:33917 (local address 127.3.89.194:41277)
0809 15:35:54.391803 (+ 442us) negotiation.cc:107] Waiting for socket to connect
0809 15:35:54.391844 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0809 15:35:54.393158 (+ 1314us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 15:35:54.393710 (+ 552us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 15:35:54.393716 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 15:35:54.394059 (+ 343us) client_negotiation.cc:189] Negotiated authn=SASL
0809 15:35:54.394515 (+ 456us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:35:54.394528 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:35:54.395408 (+ 880us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:35:54.395411 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:35:54.396322 (+ 911us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:35:54.396331 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:35:54.397041 (+ 710us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:35:54.397046 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:35:54.397250 (+ 204us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:35:54.397922 (+ 672us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 15:35:54.397938 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0809 15:35:54.399434 (+ 1496us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 15:35:54.401390 (+ 1956us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 15:35:54.401399 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 15:35:54.401416 (+ 17us) client_negotiation.cc:760] calling sasl_client_step()
0809 15:35:54.401809 (+ 393us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 15:35:54.402091 (+ 282us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 15:35:54.402094 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 15:35:54.402099 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0809 15:35:54.402280 (+ 181us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 15:35:54.402757 (+ 477us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 15:35:54.402761 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 15:35:54.403027 (+ 266us) client_negotiation.cc:769] Sending connection context
0809 15:35:54.403187 (+ 160us) client_negotiation.cc:240] Negotiation successful
0809 15:35:54.403426 (+ 239us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":273,"thread_start_us":128,"threads_started":1}
I20260809 15:35:54.405447 7640 heartbeater.cc:347] Connected to a master server at 127.3.89.254:33917
I20260809 15:35:54.405701 7640 heartbeater.cc:464] Registering TS with master...
I20260809 15:35:54.406162 7640 heartbeater.cc:511] Master 127.3.89.254:33917 requested a full tablet report, sending...
I20260809 15:35:54.407218 7295 ts_manager.cc:194] Registered new tserver with Master: 75bc8448336b42fca99c65d4b766eeab (127.3.89.194:44065)
I20260809 15:35:54.407821 7295 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.3.89.194@KRBTEST.COM'} at 127.3.89.194:41277
WARNING: no policy specified for kudu/127.3.89.195@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.3.89.195@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.3.89.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786289732823074-3431-0/krb5kdc/kudu_127.3.89.195.keytab.
Entry for principal kudu/127.3.89.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786289732823074-3431-0/krb5kdc/kudu_127.3.89.195.keytab.
I20260809 15:35:54.428203 3431 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskm314YR/build/debug/bin/kudu
/tmp/dist-test-taskm314YR/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786289732823074-3431-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786289732823074-3431-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786289732823074-3431-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786289732823074-3431-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.3.89.195:0
--local_ip_for_outbound_sockets=127.3.89.195
--webserver_interface=127.3.89.195
--webserver_port=0
--tserver_master_addrs=127.3.89.254:33917
--keytab_file=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786289732823074-3431-0/krb5kdc/kudu_127.3.89.195.keytab
--principal=kudu/127.3.89.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.3.89.212:46551
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786289732823074-3431-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786289732823074-3431-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786289732823074-3431-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 15:35:54.546339 7645 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 15:35:54.546600 7645 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 15:35:54.546684 7645 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 15:35:54.550792 7645 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 15:35:54.550882 7645 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 15:35:54.551005 7645 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.3.89.195
I20260809 15:35:54.556181 7645 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.3.89.212:46551
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786289732823074-3431-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786289732823074-3431-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786289732823074-3431-0/krb5kdc/kudu_127.3.89.195.keytab
--principal=kudu/127.3.89.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.3.89.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786289732823074-3431-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.3.89.195
--webserver_port=0
--tserver_master_addrs=127.3.89.254:33917
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.7645
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.3.89.195
--log_dir=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786289732823074-3431-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 09 Aug 2026 14:46:55 UTC on ccycloud.quasar-yswkmf.root.comops.site
build id 4576540
I20260809 15:35:54.557175 7645 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 15:35:54.558161 7645 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 15:35:54.567764 7653 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 15:35:54.567821 7650 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 15:35:54.567886 7651 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 15:35:54.568197 7645 server_base.cc:1034] running on GCE node
I20260809 15:35:54.568584 7645 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 15:35:54.569216 7645 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 15:35:54.570405 7645 hybrid_clock.cc:648] HybridClock initialized: now 1786289754570386 us; error 29 us; skew 500 ppm
Aug 09 15:35:54 dist-test-slave-2w3w krb5kdc[7247](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786289754, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.89.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 15:35:54.573364 7645 init.cc:377] Logged in from keytab as kudu/127.3.89.195@KRBTEST.COM (short username kudu)
I20260809 15:35:54.574210 7645 webserver.cc:460] Webserver started at http://127.3.89.195:39059/ using document root <none> and password file <none>
I20260809 15:35:54.574796 7645 fs_manager.cc:359] Metadata directory not provided
I20260809 15:35:54.574890 7645 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 15:35:54.575179 7645 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 15:35:54.577199 7645 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786289732823074-3431-0/minicluster-data/ts-2/data/instance:
uuid: "7dd58f3a76304b51aaf3e7c7c3073990"
format_stamp: "Formatted at 2026-08-09 15:35:54 on dist-test-slave-2w3w"
server_key: "ee097015fe8cc4f66901c487d7cbbad1"
server_key_iv: "4017744eff7f0f55d37f34fec57349dc"
server_key_version: "encryptionkey@0"
I20260809 15:35:54.577775 7645 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786289732823074-3431-0/minicluster-data/ts-2/wal/instance:
uuid: "7dd58f3a76304b51aaf3e7c7c3073990"
format_stamp: "Formatted at 2026-08-09 15:35:54 on dist-test-slave-2w3w"
server_key: "ee097015fe8cc4f66901c487d7cbbad1"
server_key_iv: "4017744eff7f0f55d37f34fec57349dc"
server_key_version: "encryptionkey@0"
I20260809 15:35:54.581427 7645 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.005s sys 0.000s
I20260809 15:35:54.583828 7661 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 15:35:54.585003 7645 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260809 15:35:54.585139 7645 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786289732823074-3431-0/minicluster-data/ts-2/data,/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786289732823074-3431-0/minicluster-data/ts-2/wal
uuid: "7dd58f3a76304b51aaf3e7c7c3073990"
format_stamp: "Formatted at 2026-08-09 15:35:54 on dist-test-slave-2w3w"
server_key: "ee097015fe8cc4f66901c487d7cbbad1"
server_key_iv: "4017744eff7f0f55d37f34fec57349dc"
server_key_version: "encryptionkey@0"
I20260809 15:35:54.585249 7645 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786289732823074-3431-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786289732823074-3431-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786289732823074-3431-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 15:35:54.598061 7645 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 15:35:54.601374 7645 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 15:35:54.601584 7645 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 15:35:54.602193 7645 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 15:35:54.603107 7645 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 15:35:54.603178 7645 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 15:35:54.603245 7645 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 15:35:54.603297 7645 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 15:35:54.611907 7645 rpc_server.cc:307] RPC server started. Bound to: 127.3.89.195:42781
I20260809 15:35:54.611974 7779 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.3.89.195:42781 every 8 connection(s)
I20260809 15:35:54.612880 7645 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786289732823074-3431-0/minicluster-data/ts-2/data/info.pb
I20260809 15:35:54.615197 3431 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskm314YR/build/debug/bin/kudu as pid 7645
I20260809 15:35:54.615358 3431 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786289732823074-3431-0/minicluster-data/ts-2/wal/instance
I20260809 15:35:54.615700 3431 external_mini_cluster.cc:1428] Setting key c4235a3fd4a6eedc432beeadfde190fb
Aug 09 15:35:54 dist-test-slave-2w3w krb5kdc[7247](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786289754, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.89.195@KRBTEST.COM for kudu/127.3.89.254@KRBTEST.COM
I20260809 15:35:54.626955 7338 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:35:54.614671 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.89.195:40247 (local address 127.3.89.254:33917)
0809 15:35:54.614828 (+ 157us) server_negotiation.cc:206] Beginning negotiation
0809 15:35:54.614832 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 15:35:54.615776 (+ 944us) server_negotiation.cc:407] Connection header received
0809 15:35:54.617481 (+ 1705us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 15:35:54.617484 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 15:35:54.617553 (+ 69us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 15:35:54.617657 (+ 104us) server_negotiation.cc:226] Negotiated authn=SASL
0809 15:35:54.618999 (+ 1342us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:35:54.619577 (+ 578us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:35:54.620731 (+ 1154us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:35:54.620966 (+ 235us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:35:54.621071 (+ 105us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:35:54.623503 (+ 2432us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 15:35:54.623507 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 15:35:54.623509 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 15:35:54.623542 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0809 15:35:54.624977 (+ 1435us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 15:35:54.625490 (+ 513us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 15:35:54.625495 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 15:35:54.625497 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 15:35:54.625581 (+ 84us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 15:35:54.625935 (+ 354us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 15:35:54.625938 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 15:35:54.625940 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 15:35:54.626115 (+ 175us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 15:35:54.626221 (+ 106us) server_negotiation.cc:1035] Waiting for connection context
0809 15:35:54.626670 (+ 449us) server_negotiation.cc:299] Negotiation successful
0809 15:35:54.626789 (+ 119us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":60}
I20260809 15:35:54.627727 7782 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:35:54.615059 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.89.254:33917 (local address 127.3.89.195:40247)
0809 15:35:54.615624 (+ 565us) negotiation.cc:107] Waiting for socket to connect
0809 15:35:54.615665 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0809 15:35:54.617283 (+ 1618us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 15:35:54.617839 (+ 556us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 15:35:54.617846 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 15:35:54.618287 (+ 441us) client_negotiation.cc:189] Negotiated authn=SASL
0809 15:35:54.618839 (+ 552us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:35:54.618855 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:35:54.619707 (+ 852us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:35:54.619711 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:35:54.620598 (+ 887us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:35:54.620607 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:35:54.621118 (+ 511us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:35:54.621122 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:35:54.621287 (+ 165us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:35:54.621893 (+ 606us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 15:35:54.621908 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0809 15:35:54.623344 (+ 1436us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 15:35:54.625108 (+ 1764us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 15:35:54.625113 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 15:35:54.625126 (+ 13us) client_negotiation.cc:760] calling sasl_client_step()
0809 15:35:54.625385 (+ 259us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 15:35:54.625676 (+ 291us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 15:35:54.625680 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 15:35:54.625682 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 15:35:54.625839 (+ 157us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 15:35:54.626219 (+ 380us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 15:35:54.626226 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 15:35:54.626446 (+ 220us) client_negotiation.cc:769] Sending connection context
0809 15:35:54.626626 (+ 180us) client_negotiation.cc:240] Negotiation successful
0809 15:35:54.626926 (+ 300us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":371,"thread_start_us":153,"threads_started":1}
I20260809 15:35:54.628841 7780 heartbeater.cc:347] Connected to a master server at 127.3.89.254:33917
I20260809 15:35:54.629076 7780 heartbeater.cc:464] Registering TS with master...
I20260809 15:35:54.629524 7780 heartbeater.cc:511] Master 127.3.89.254:33917 requested a full tablet report, sending...
I20260809 15:35:54.630537 7294 ts_manager.cc:194] Registered new tserver with Master: 7dd58f3a76304b51aaf3e7c7c3073990 (127.3.89.195:42781)
I20260809 15:35:54.631233 7294 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.3.89.195@KRBTEST.COM'} at 127.3.89.195:40247
I20260809 15:35:54.641080 3431 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Aug 09 15:35:54 dist-test-slave-2w3w krb5kdc[7247](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786289754, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Aug 09 15:35:54 dist-test-slave-2w3w krb5kdc[7247](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786289754, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.3.89.193@KRBTEST.COM
I20260809 15:35:54.667765 7787 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:35:54.658427 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48588 (local address 127.3.89.193:46755)
0809 15:35:54.658735 (+ 308us) server_negotiation.cc:206] Beginning negotiation
0809 15:35:54.658775 (+ 40us) server_negotiation.cc:399] Waiting for connection header
0809 15:35:54.658801 (+ 26us) server_negotiation.cc:407] Connection header received
0809 15:35:54.658848 (+ 47us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 15:35:54.658852 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 15:35:54.659085 (+ 233us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 15:35:54.659283 (+ 198us) server_negotiation.cc:226] Negotiated authn=SASL
0809 15:35:54.660046 (+ 763us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:35:54.660956 (+ 910us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:35:54.661830 (+ 874us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:35:54.662186 (+ 356us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:35:54.662270 (+ 84us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:35:54.664644 (+ 2374us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 15:35:54.664653 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 15:35:54.664665 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 15:35:54.664687 (+ 22us) server_negotiation.cc:922] calling sasl_server_start()
0809 15:35:54.666195 (+ 1508us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 15:35:54.666577 (+ 382us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 15:35:54.666582 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 15:35:54.666583 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 15:35:54.666658 (+ 75us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 15:35:54.666883 (+ 225us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 15:35:54.666886 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 15:35:54.666887 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 15:35:54.667172 (+ 285us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 15:35:54.667329 (+ 157us) server_negotiation.cc:1035] Waiting for connection context
0809 15:35:54.667485 (+ 156us) server_negotiation.cc:299] Negotiation successful
0809 15:35:54.667621 (+ 136us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":207,"thread_start_us":134,"threads_started":1}
Aug 09 15:35:54 dist-test-slave-2w3w krb5kdc[7247](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786289754, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.3.89.254@KRBTEST.COM
I20260809 15:35:54.678202 7338 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:35:54.670469 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56112 (local address 127.3.89.254:33917)
0809 15:35:54.670609 (+ 140us) server_negotiation.cc:206] Beginning negotiation
0809 15:35:54.670612 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0809 15:35:54.670625 (+ 13us) server_negotiation.cc:407] Connection header received
0809 15:35:54.670745 (+ 120us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 15:35:54.670747 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 15:35:54.670808 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 15:35:54.670874 (+ 66us) server_negotiation.cc:226] Negotiated authn=SASL
0809 15:35:54.671468 (+ 594us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:35:54.671969 (+ 501us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:35:54.672683 (+ 714us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:35:54.672901 (+ 218us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:35:54.672976 (+ 75us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:35:54.675250 (+ 2274us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 15:35:54.675253 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 15:35:54.675255 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 15:35:54.675277 (+ 22us) server_negotiation.cc:922] calling sasl_server_start()
0809 15:35:54.676479 (+ 1202us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 15:35:54.676958 (+ 479us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 15:35:54.676962 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 15:35:54.676964 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 15:35:54.677044 (+ 80us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 15:35:54.677334 (+ 290us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 15:35:54.677338 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 15:35:54.677339 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 15:35:54.677556 (+ 217us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 15:35:54.677699 (+ 143us) server_negotiation.cc:1035] Waiting for connection context
0809 15:35:54.677926 (+ 227us) server_negotiation.cc:299] Negotiation successful
0809 15:35:54.678046 (+ 120us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":62}
W20260809 15:35:54.678601 7294 server_base.cc:1116] Unauthorized access attempt to method kudu.master.MasterService.TSHeartbeat from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:56112
I20260809 15:35:54.679271 3431 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskm314YR/build/debug/bin/kudu with pid 7363
I20260809 15:35:54.686149 3431 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskm314YR/build/debug/bin/kudu with pid 7505
I20260809 15:35:54.692894 3431 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskm314YR/build/debug/bin/kudu with pid 7645
I20260809 15:35:54.701799 3431 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskm314YR/build/debug/bin/kudu with pid 7261
2026-08-09T15:35:54Z chronyd exiting
[ OK ] SecurityITest.TestAuthorizedSuperuser (2207 ms)
[ RUN ] SecurityITest.TestDisableWebUI
Loading random data
Initializing database '/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786289732823074-3431-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 15:35:54 dist-test-slave-2w3w krb5kdc[7792](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 15:35:54 dist-test-slave-2w3w krb5kdc[7792](info): set up 2 sockets
Aug 09 15:35:54 dist-test-slave-2w3w krb5kdc[7792](info): commencing operation
krb5kdc: starting...
W20260809 15:35:55.788718 3431 mini_kdc.cc:121] Time spent starting KDC: real 1.064s user 0.003s sys 0.004s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786289732823074-3431-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786289732823074-3431-0/krb5kdc/test-user.keytab.
Aug 09 15:35:55 dist-test-slave-2w3w krb5kdc[7792](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786289755, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T15:35:55Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T15:35:55Z Disabled control of system clock
WARNING: no policy specified for kudu/127.3.89.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.3.89.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.3.89.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786289732823074-3431-0/krb5kdc/kudu_127.3.89.254.keytab.
Entry for principal kudu/127.3.89.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786289732823074-3431-0/krb5kdc/kudu_127.3.89.254.keytab.
I20260809 15:35:55.912194 3431 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskm314YR/build/debug/bin/kudu
/tmp/dist-test-taskm314YR/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786289732823074-3431-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786289732823074-3431-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786289732823074-3431-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786289732823074-3431-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.3.89.254:45901
--webserver_interface=127.3.89.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786289732823074-3431-0/krb5kdc/kudu_127.3.89.254.keytab
--principal=kudu/127.3.89.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.3.89.212:43745
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.3.89.254:45901
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--webserver_enabled=0 with env {KRB5CCNAME=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786289732823074-3431-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786289732823074-3431-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786289732823074-3431-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 15:35:56.023012 7806 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 15:35:56.023274 7806 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 15:35:56.023317 7806 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 15:35:56.027531 7806 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 15:35:56.027585 7806 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 15:35:56.027604 7806 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 15:35:56.027621 7806 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 15:35:56.027674 7806 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 15:35:56.032790 7806 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.3.89.212:43745
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786289732823074-3431-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786289732823074-3431-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.3.89.254:45901
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786289732823074-3431-0/krb5kdc/kudu_127.3.89.254.keytab
--principal=kudu/127.3.89.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.3.89.254:45901
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--webserver_enabled=false
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786289732823074-3431-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.3.89.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.7806
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786289732823074-3431-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 09 Aug 2026 14:46:55 UTC on ccycloud.quasar-yswkmf.root.comops.site
build id 4576540
I20260809 15:35:56.033829 7806 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 15:35:56.034894 7806 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 15:35:56.043190 7814 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 15:35:56.043241 7811 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 15:35:56.043203 7806 server_base.cc:1034] running on GCE node
W20260809 15:35:56.043180 7812 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 15:35:56.044063 7806 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 15:35:56.045223 7806 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 15:35:56.046420 7806 hybrid_clock.cc:648] HybridClock initialized: now 1786289756046409 us; error 31 us; skew 500 ppm
Aug 09 15:35:56 dist-test-slave-2w3w krb5kdc[7792](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786289756, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.89.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 15:35:56.049270 7806 init.cc:377] Logged in from keytab as kudu/127.3.89.254@KRBTEST.COM (short username kudu)
I20260809 15:35:56.049923 7806 fs_manager.cc:359] Metadata directory not provided
I20260809 15:35:56.049985 7806 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 15:35:56.050144 7806 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 15:35:56.052156 7806 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786289732823074-3431-0/minicluster-data/master-0/data/instance:
uuid: "e41b0a7c3aa14466942eac2d1058406a"
format_stamp: "Formatted at 2026-08-09 15:35:56 on dist-test-slave-2w3w"
server_key: "cc1c595210dbd9ed5ae149342d8ec6c8"
server_key_iv: "11a5b20f35cec9e88bdebab431f58f80"
server_key_version: "encryptionkey@0"
I20260809 15:35:56.052687 7806 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786289732823074-3431-0/minicluster-data/master-0/wal/instance:
uuid: "e41b0a7c3aa14466942eac2d1058406a"
format_stamp: "Formatted at 2026-08-09 15:35:56 on dist-test-slave-2w3w"
server_key: "cc1c595210dbd9ed5ae149342d8ec6c8"
server_key_iv: "11a5b20f35cec9e88bdebab431f58f80"
server_key_version: "encryptionkey@0"
I20260809 15:35:56.056274 7806 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.004s sys 0.000s
I20260809 15:35:56.058940 7820 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 15:35:56.060249 7806 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260809 15:35:56.060402 7806 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786289732823074-3431-0/minicluster-data/master-0/data,/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786289732823074-3431-0/minicluster-data/master-0/wal
uuid: "e41b0a7c3aa14466942eac2d1058406a"
format_stamp: "Formatted at 2026-08-09 15:35:56 on dist-test-slave-2w3w"
server_key: "cc1c595210dbd9ed5ae149342d8ec6c8"
server_key_iv: "11a5b20f35cec9e88bdebab431f58f80"
server_key_version: "encryptionkey@0"
I20260809 15:35:56.060514 7806 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786289732823074-3431-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786289732823074-3431-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786289732823074-3431-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 15:35:56.084925 7806 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 15:35:56.087952 7806 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 15:35:56.088172 7806 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 15:35:56.095166 7806 rpc_server.cc:307] RPC server started. Bound to: 127.3.89.254:45901
I20260809 15:35:56.095451 7877 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.3.89.254:45901 every 8 connection(s)
I20260809 15:35:56.095861 7806 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786289732823074-3431-0/minicluster-data/master-0/data/info.pb
I20260809 15:35:56.098606 7878 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 15:35:56.098662 3431 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskm314YR/build/debug/bin/kudu as pid 7806
I20260809 15:35:56.098814 3431 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786289732823074-3431-0/minicluster-data/master-0/wal/instance
I20260809 15:35:56.099148 3431 external_mini_cluster.cc:1428] Setting key e63673783af1f3c770cb631e07a4ece2
I20260809 15:35:56.105863 7878 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e41b0a7c3aa14466942eac2d1058406a: Bootstrap starting.
Aug 09 15:35:56 dist-test-slave-2w3w krb5kdc[7792](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786289755, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.3.89.254@KRBTEST.COM
I20260809 15:35:56.108672 7878 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P e41b0a7c3aa14466942eac2d1058406a: Neither blocks nor log segments found. Creating new log.
I20260809 15:35:56.109417 7878 log.cc:824] T 00000000000000000000000000000000 P e41b0a7c3aa14466942eac2d1058406a: Log is configured to *not* fsync() on all Append() calls
I20260809 15:35:56.111526 7878 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e41b0a7c3aa14466942eac2d1058406a: No bootstrap required, opened a new log
I20260809 15:35:56.114457 7881 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:35:56.100446 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36218 (local address 127.3.89.254:45901)
0809 15:35:56.100845 (+ 399us) server_negotiation.cc:206] Beginning negotiation
0809 15:35:56.100873 (+ 28us) server_negotiation.cc:399] Waiting for connection header
0809 15:35:56.100908 (+ 35us) server_negotiation.cc:407] Connection header received
0809 15:35:56.102101 (+ 1193us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 15:35:56.102129 (+ 28us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 15:35:56.102535 (+ 406us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 15:35:56.102809 (+ 274us) server_negotiation.cc:226] Negotiated authn=SASL
0809 15:35:56.103754 (+ 945us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:35:56.104740 (+ 986us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:35:56.105884 (+ 1144us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:35:56.106386 (+ 502us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:35:56.106511 (+ 125us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:35:56.109269 (+ 2758us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 15:35:56.109274 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 15:35:56.109290 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 15:35:56.109317 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0809 15:35:56.111265 (+ 1948us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 15:35:56.111734 (+ 469us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 15:35:56.111740 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 15:35:56.111746 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0809 15:35:56.111896 (+ 150us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 15:35:56.112186 (+ 290us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 15:35:56.112189 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 15:35:56.112191 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 15:35:56.112686 (+ 495us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 15:35:56.112874 (+ 188us) server_negotiation.cc:1035] Waiting for connection context
0809 15:35:56.113118 (+ 244us) server_negotiation.cc:299] Negotiation successful
0809 15:35:56.113412 (+ 294us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":256,"thread_start_us":131,"threads_started":1}
I20260809 15:35:56.115993 7878 raft_consensus.cc:348] T 00000000000000000000000000000000 P e41b0a7c3aa14466942eac2d1058406a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e41b0a7c3aa14466942eac2d1058406a" member_type: VOTER last_known_addr { host: "127.3.89.254" port: 45901 } }
I20260809 15:35:56.116206 7878 raft_consensus.cc:374] T 00000000000000000000000000000000 P e41b0a7c3aa14466942eac2d1058406a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 15:35:56.116263 7878 raft_consensus.cc:729] T 00000000000000000000000000000000 P e41b0a7c3aa14466942eac2d1058406a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e41b0a7c3aa14466942eac2d1058406a, State: Initialized, Role: FOLLOWER
I20260809 15:35:56.116667 7878 consensus_queue.cc:260] T 00000000000000000000000000000000 P e41b0a7c3aa14466942eac2d1058406a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e41b0a7c3aa14466942eac2d1058406a" member_type: VOTER last_known_addr { host: "127.3.89.254" port: 45901 } }
I20260809 15:35:56.116819 7878 raft_consensus.cc:388] T 00000000000000000000000000000000 P e41b0a7c3aa14466942eac2d1058406a [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 15:35:56.116902 7878 raft_consensus.cc:482] T 00000000000000000000000000000000 P e41b0a7c3aa14466942eac2d1058406a [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 15:35:56.117036 7878 raft_consensus.cc:3037] T 00000000000000000000000000000000 P e41b0a7c3aa14466942eac2d1058406a [term 0 FOLLOWER]: Advancing to term 1
I20260809 15:35:56.117954 7878 raft_consensus.cc:504] T 00000000000000000000000000000000 P e41b0a7c3aa14466942eac2d1058406a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e41b0a7c3aa14466942eac2d1058406a" member_type: VOTER last_known_addr { host: "127.3.89.254" port: 45901 } }
I20260809 15:35:56.118287 7878 leader_election.cc:302] T 00000000000000000000000000000000 P e41b0a7c3aa14466942eac2d1058406a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: e41b0a7c3aa14466942eac2d1058406a; no voters:
I20260809 15:35:56.118623 7878 leader_election.cc:288] T 00000000000000000000000000000000 P e41b0a7c3aa14466942eac2d1058406a [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 15:35:56.118703 7883 raft_consensus.cc:2781] T 00000000000000000000000000000000 P e41b0a7c3aa14466942eac2d1058406a [term 1 FOLLOWER]: Leader election won for term 1
I20260809 15:35:56.118901 7883 raft_consensus.cc:686] T 00000000000000000000000000000000 P e41b0a7c3aa14466942eac2d1058406a [term 1 LEADER]: Becoming Leader. State: Replica: e41b0a7c3aa14466942eac2d1058406a, State: Running, Role: LEADER
I20260809 15:35:56.119215 7883 consensus_queue.cc:237] T 00000000000000000000000000000000 P e41b0a7c3aa14466942eac2d1058406a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e41b0a7c3aa14466942eac2d1058406a" member_type: VOTER last_known_addr { host: "127.3.89.254" port: 45901 } }
I20260809 15:35:56.119949 7878 sys_catalog.cc:564] T 00000000000000000000000000000000 P e41b0a7c3aa14466942eac2d1058406a [sys.catalog]: configured and running, proceeding with master startup.
I20260809 15:35:56.121160 7884 sys_catalog.cc:455] T 00000000000000000000000000000000 P e41b0a7c3aa14466942eac2d1058406a [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "e41b0a7c3aa14466942eac2d1058406a" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e41b0a7c3aa14466942eac2d1058406a" member_type: VOTER last_known_addr { host: "127.3.89.254" port: 45901 } } }
I20260809 15:35:56.121282 7884 sys_catalog.cc:458] T 00000000000000000000000000000000 P e41b0a7c3aa14466942eac2d1058406a [sys.catalog]: This master's current role is: LEADER
I20260809 15:35:56.121688 7885 sys_catalog.cc:455] T 00000000000000000000000000000000 P e41b0a7c3aa14466942eac2d1058406a [sys.catalog]: SysCatalogTable state changed. Reason: New leader e41b0a7c3aa14466942eac2d1058406a. Latest consensus state: current_term: 1 leader_uuid: "e41b0a7c3aa14466942eac2d1058406a" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e41b0a7c3aa14466942eac2d1058406a" member_type: VOTER last_known_addr { host: "127.3.89.254" port: 45901 } } }
I20260809 15:35:56.121800 7885 sys_catalog.cc:458] T 00000000000000000000000000000000 P e41b0a7c3aa14466942eac2d1058406a [sys.catalog]: This master's current role is: LEADER
W20260809 15:35:56.124794 7899 catalog_manager.cc:1558] T 00000000000000000000000000000000 P e41b0a7c3aa14466942eac2d1058406a: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260809 15:35:56.124914 7899 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260809 15:35:56.124998 7892 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 15:35:56.125703 7892 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 15:35:56.131488 7892 catalog_manager.cc:1347] Generated new cluster ID: 1a4be089d5f844ab81073be5868c9f56
I20260809 15:35:56.131564 7892 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 15:35:56.146226 7892 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 15:35:56.147331 7892 catalog_manager.cc:1504] Loading token signing keys...
I20260809 15:35:56.163358 7892 catalog_manager.cc:5868] T 00000000000000000000000000000000 P e41b0a7c3aa14466942eac2d1058406a: Generated new TSK 0
I20260809 15:35:56.164057 7892 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.3.89.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.3.89.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.3.89.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786289732823074-3431-0/krb5kdc/kudu_127.3.89.193.keytab.
Entry for principal kudu/127.3.89.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786289732823074-3431-0/krb5kdc/kudu_127.3.89.193.keytab.
I20260809 15:35:56.198819 3431 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskm314YR/build/debug/bin/kudu
/tmp/dist-test-taskm314YR/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786289732823074-3431-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786289732823074-3431-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786289732823074-3431-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786289732823074-3431-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.3.89.193:0
--local_ip_for_outbound_sockets=127.3.89.193
--webserver_interface=127.3.89.193
--webserver_port=0
--tserver_master_addrs=127.3.89.254:45901
--keytab_file=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786289732823074-3431-0/krb5kdc/kudu_127.3.89.193.keytab
--principal=kudu/127.3.89.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.3.89.212:43745
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--webserver_enabled=0 with env {KRB5CCNAME=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786289732823074-3431-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786289732823074-3431-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786289732823074-3431-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 15:35:56.308637 7904 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 15:35:56.308884 7904 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 15:35:56.308928 7904 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 15:35:56.312785 7904 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 15:35:56.312831 7904 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 15:35:56.312888 7904 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.3.89.193
I20260809 15:35:56.318120 7904 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.3.89.212:43745
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786289732823074-3431-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786289732823074-3431-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786289732823074-3431-0/krb5kdc/kudu_127.3.89.193.keytab
--principal=kudu/127.3.89.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.3.89.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--webserver_enabled=false
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786289732823074-3431-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.3.89.193
--webserver_port=0
--tserver_master_addrs=127.3.89.254:45901
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.7904
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.3.89.193
--log_dir=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786289732823074-3431-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 09 Aug 2026 14:46:55 UTC on ccycloud.quasar-yswkmf.root.comops.site
build id 4576540
I20260809 15:35:56.319073 7904 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 15:35:56.320106 7904 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 15:35:56.329522 7912 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 15:35:56.329629 7904 server_base.cc:1034] running on GCE node
W20260809 15:35:56.329545 7909 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 15:35:56.329744 7910 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 15:35:56.330173 7904 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 15:35:56.330768 7904 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 15:35:56.331945 7904 hybrid_clock.cc:648] HybridClock initialized: now 1786289756331911 us; error 41 us; skew 500 ppm
Aug 09 15:35:56 dist-test-slave-2w3w krb5kdc[7792](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786289756, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.89.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 15:35:56.335094 7904 init.cc:377] Logged in from keytab as kudu/127.3.89.193@KRBTEST.COM (short username kudu)
I20260809 15:35:56.335754 7904 fs_manager.cc:359] Metadata directory not provided
I20260809 15:35:56.335826 7904 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 15:35:56.336057 7904 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 15:35:56.337987 7904 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786289732823074-3431-0/minicluster-data/ts-0/data/instance:
uuid: "52ce2fe4c3b64218b0a11bc0f07fc6de"
format_stamp: "Formatted at 2026-08-09 15:35:56 on dist-test-slave-2w3w"
server_key: "1224adb6ee463d779b1591e1b51b96e8"
server_key_iv: "11c31f9d96fb29cd35752f054710c762"
server_key_version: "encryptionkey@0"
I20260809 15:35:56.338492 7904 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786289732823074-3431-0/minicluster-data/ts-0/wal/instance:
uuid: "52ce2fe4c3b64218b0a11bc0f07fc6de"
format_stamp: "Formatted at 2026-08-09 15:35:56 on dist-test-slave-2w3w"
server_key: "1224adb6ee463d779b1591e1b51b96e8"
server_key_iv: "11c31f9d96fb29cd35752f054710c762"
server_key_version: "encryptionkey@0"
I20260809 15:35:56.342021 7904 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.006s sys 0.000s
I20260809 15:35:56.344354 7918 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 15:35:56.345564 7904 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260809 15:35:56.345698 7904 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786289732823074-3431-0/minicluster-data/ts-0/data,/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786289732823074-3431-0/minicluster-data/ts-0/wal
uuid: "52ce2fe4c3b64218b0a11bc0f07fc6de"
format_stamp: "Formatted at 2026-08-09 15:35:56 on dist-test-slave-2w3w"
server_key: "1224adb6ee463d779b1591e1b51b96e8"
server_key_iv: "11c31f9d96fb29cd35752f054710c762"
server_key_version: "encryptionkey@0"
I20260809 15:35:56.345830 7904 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786289732823074-3431-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786289732823074-3431-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786289732823074-3431-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 15:35:56.373011 7904 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 15:35:56.376088 7904 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 15:35:56.376296 7904 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 15:35:56.376917 7904 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 15:35:56.377771 7904 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 15:35:56.377827 7904 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 15:35:56.377916 7904 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 15:35:56.377949 7904 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 15:35:56.386761 7904 rpc_server.cc:307] RPC server started. Bound to: 127.3.89.193:43777
I20260809 15:35:56.386785 8037 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.3.89.193:43777 every 8 connection(s)
I20260809 15:35:56.387425 7904 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786289732823074-3431-0/minicluster-data/ts-0/data/info.pb
I20260809 15:35:56.395576 3431 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskm314YR/build/debug/bin/kudu as pid 7904
I20260809 15:35:56.395716 3431 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786289732823074-3431-0/minicluster-data/ts-0/wal/instance
I20260809 15:35:56.396013 3431 external_mini_cluster.cc:1428] Setting key 380e879cc46c175db13fbbcb9f31bcc2
Aug 09 15:35:56 dist-test-slave-2w3w krb5kdc[7792](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786289756, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.89.193@KRBTEST.COM for kudu/127.3.89.254@KRBTEST.COM
I20260809 15:35:56.401650 7881 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:35:56.389167 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.89.193:34027 (local address 127.3.89.254:45901)
0809 15:35:56.389306 (+ 139us) server_negotiation.cc:206] Beginning negotiation
0809 15:35:56.389310 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 15:35:56.390098 (+ 788us) server_negotiation.cc:407] Connection header received
0809 15:35:56.391457 (+ 1359us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 15:35:56.391462 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 15:35:56.391521 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 15:35:56.391606 (+ 85us) server_negotiation.cc:226] Negotiated authn=SASL
0809 15:35:56.392878 (+ 1272us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:35:56.393447 (+ 569us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:35:56.394615 (+ 1168us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:35:56.394876 (+ 261us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:35:56.395021 (+ 145us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:35:56.397739 (+ 2718us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 15:35:56.397742 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 15:35:56.397744 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 15:35:56.397768 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0809 15:35:56.399255 (+ 1487us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 15:35:56.399924 (+ 669us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 15:35:56.399930 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 15:35:56.399934 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 15:35:56.400038 (+ 104us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 15:35:56.400373 (+ 335us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 15:35:56.400378 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 15:35:56.400380 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 15:35:56.400653 (+ 273us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 15:35:56.400761 (+ 108us) server_negotiation.cc:1035] Waiting for connection context
0809 15:35:56.401254 (+ 493us) server_negotiation.cc:299] Negotiation successful
0809 15:35:56.401396 (+ 142us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":54}
I20260809 15:35:56.402262 8040 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:35:56.389521 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.89.254:45901 (local address 127.3.89.193:34027)
0809 15:35:56.389943 (+ 422us) negotiation.cc:107] Waiting for socket to connect
0809 15:35:56.389989 (+ 46us) client_negotiation.cc:174] Beginning negotiation
0809 15:35:56.391245 (+ 1256us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 15:35:56.391770 (+ 525us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 15:35:56.391777 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 15:35:56.392131 (+ 354us) client_negotiation.cc:189] Negotiated authn=SASL
0809 15:35:56.392630 (+ 499us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:35:56.392644 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:35:56.393587 (+ 943us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:35:56.393591 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:35:56.394477 (+ 886us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:35:56.394486 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:35:56.395074 (+ 588us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:35:56.395078 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:35:56.395250 (+ 172us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:35:56.395872 (+ 622us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 15:35:56.395887 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0809 15:35:56.397556 (+ 1669us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 15:35:56.399375 (+ 1819us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 15:35:56.399383 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 15:35:56.399400 (+ 17us) client_negotiation.cc:760] calling sasl_client_step()
0809 15:35:56.399802 (+ 402us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 15:35:56.400140 (+ 338us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 15:35:56.400142 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 15:35:56.400143 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0809 15:35:56.400276 (+ 133us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 15:35:56.400760 (+ 484us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 15:35:56.400764 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 15:35:56.401014 (+ 250us) client_negotiation.cc:769] Sending connection context
0809 15:35:56.401168 (+ 154us) client_negotiation.cc:240] Negotiation successful
0809 15:35:56.401411 (+ 243us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":252,"thread_start_us":118,"threads_started":1}
I20260809 15:35:56.403592 8038 heartbeater.cc:347] Connected to a master server at 127.3.89.254:45901
I20260809 15:35:56.403849 8038 heartbeater.cc:464] Registering TS with master...
I20260809 15:35:56.404280 8038 heartbeater.cc:511] Master 127.3.89.254:45901 requested a full tablet report, sending...
I20260809 15:35:56.405733 7838 ts_manager.cc:194] Registered new tserver with Master: 52ce2fe4c3b64218b0a11bc0f07fc6de (127.3.89.193:43777)
WARNING: no policy specified for kudu/127.3.89.194@KRBTEST.COM; defaulting to no policy
I20260809 15:35:56.406924 7838 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.3.89.193@KRBTEST.COM'} at 127.3.89.193:34027
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.3.89.194@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.3.89.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786289732823074-3431-0/krb5kdc/kudu_127.3.89.194.keytab.
Entry for principal kudu/127.3.89.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786289732823074-3431-0/krb5kdc/kudu_127.3.89.194.keytab.
I20260809 15:35:56.426230 3431 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskm314YR/build/debug/bin/kudu
/tmp/dist-test-taskm314YR/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786289732823074-3431-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786289732823074-3431-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786289732823074-3431-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786289732823074-3431-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.3.89.194:0
--local_ip_for_outbound_sockets=127.3.89.194
--webserver_interface=127.3.89.194
--webserver_port=0
--tserver_master_addrs=127.3.89.254:45901
--keytab_file=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786289732823074-3431-0/krb5kdc/kudu_127.3.89.194.keytab
--principal=kudu/127.3.89.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.3.89.212:43745
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--webserver_enabled=0 with env {KRB5CCNAME=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786289732823074-3431-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786289732823074-3431-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786289732823074-3431-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 15:35:56.538831 8043 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 15:35:56.539163 8043 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 15:35:56.539213 8043 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 15:35:56.543222 8043 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 15:35:56.543274 8043 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 15:35:56.543332 8043 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.3.89.194
I20260809 15:35:56.548461 8043 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.3.89.212:43745
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786289732823074-3431-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786289732823074-3431-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786289732823074-3431-0/krb5kdc/kudu_127.3.89.194.keytab
--principal=kudu/127.3.89.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.3.89.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--webserver_enabled=false
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786289732823074-3431-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.3.89.194
--webserver_port=0
--tserver_master_addrs=127.3.89.254:45901
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.8043
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.3.89.194
--log_dir=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786289732823074-3431-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 09 Aug 2026 14:46:55 UTC on ccycloud.quasar-yswkmf.root.comops.site
build id 4576540
I20260809 15:35:56.549528 8043 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 15:35:56.550534 8043 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 15:35:56.559924 8050 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 15:35:56.559971 8049 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 15:35:56.559945 8052 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 15:35:56.560055 8043 server_base.cc:1034] running on GCE node
I20260809 15:35:56.560623 8043 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 15:35:56.561267 8043 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 15:35:56.562446 8043 hybrid_clock.cc:648] HybridClock initialized: now 1786289756562428 us; error 24 us; skew 500 ppm
Aug 09 15:35:56 dist-test-slave-2w3w krb5kdc[7792](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786289756, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.89.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 15:35:56.565387 8043 init.cc:377] Logged in from keytab as kudu/127.3.89.194@KRBTEST.COM (short username kudu)
I20260809 15:35:56.566057 8043 fs_manager.cc:359] Metadata directory not provided
I20260809 15:35:56.566119 8043 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 15:35:56.566277 8043 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 15:35:56.568291 8043 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786289732823074-3431-0/minicluster-data/ts-1/data/instance:
uuid: "c0c52efb3a8b4a5d9afa1c03c4790182"
format_stamp: "Formatted at 2026-08-09 15:35:56 on dist-test-slave-2w3w"
server_key: "b819e5f2c534f78bb813e8a484dc27b0"
server_key_iv: "4d86ca4391189ff27b136c76fd797cfa"
server_key_version: "encryptionkey@0"
I20260809 15:35:56.568822 8043 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786289732823074-3431-0/minicluster-data/ts-1/wal/instance:
uuid: "c0c52efb3a8b4a5d9afa1c03c4790182"
format_stamp: "Formatted at 2026-08-09 15:35:56 on dist-test-slave-2w3w"
server_key: "b819e5f2c534f78bb813e8a484dc27b0"
server_key_iv: "4d86ca4391189ff27b136c76fd797cfa"
server_key_version: "encryptionkey@0"
I20260809 15:35:56.572424 8043 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.004s sys 0.000s
I20260809 15:35:56.574831 8058 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 15:35:56.576085 8043 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.001s
I20260809 15:35:56.576213 8043 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786289732823074-3431-0/minicluster-data/ts-1/data,/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786289732823074-3431-0/minicluster-data/ts-1/wal
uuid: "c0c52efb3a8b4a5d9afa1c03c4790182"
format_stamp: "Formatted at 2026-08-09 15:35:56 on dist-test-slave-2w3w"
server_key: "b819e5f2c534f78bb813e8a484dc27b0"
server_key_iv: "4d86ca4391189ff27b136c76fd797cfa"
server_key_version: "encryptionkey@0"
I20260809 15:35:56.576318 8043 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786289732823074-3431-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786289732823074-3431-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786289732823074-3431-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 15:35:56.591621 8043 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 15:35:56.595018 8043 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 15:35:56.595235 8043 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 15:35:56.595862 8043 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 15:35:56.596702 8043 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 15:35:56.596776 8043 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 15:35:56.596853 8043 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 15:35:56.596897 8043 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 15:35:56.605705 8043 rpc_server.cc:307] RPC server started. Bound to: 127.3.89.194:44803
I20260809 15:35:56.605769 8180 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.3.89.194:44803 every 8 connection(s)
I20260809 15:35:56.606338 8043 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786289732823074-3431-0/minicluster-data/ts-1/data/info.pb
I20260809 15:35:56.612831 3431 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskm314YR/build/debug/bin/kudu as pid 8043
I20260809 15:35:56.612973 3431 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786289732823074-3431-0/minicluster-data/ts-1/wal/instance
I20260809 15:35:56.613333 3431 external_mini_cluster.cc:1428] Setting key 9233cfd8ef1edda19239c28eaef60d9a
Aug 09 15:35:56 dist-test-slave-2w3w krb5kdc[7792](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786289756, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.89.194@KRBTEST.COM for kudu/127.3.89.254@KRBTEST.COM
I20260809 15:35:56.620136 7881 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:35:56.608049 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.89.194:47781 (local address 127.3.89.254:45901)
0809 15:35:56.608218 (+ 169us) server_negotiation.cc:206] Beginning negotiation
0809 15:35:56.608223 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 15:35:56.609019 (+ 796us) server_negotiation.cc:407] Connection header received
0809 15:35:56.610297 (+ 1278us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 15:35:56.610300 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 15:35:56.610346 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 15:35:56.610421 (+ 75us) server_negotiation.cc:226] Negotiated authn=SASL
0809 15:35:56.611566 (+ 1145us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:35:56.612097 (+ 531us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:35:56.613303 (+ 1206us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:35:56.613620 (+ 317us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:35:56.613747 (+ 127us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:35:56.616382 (+ 2635us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 15:35:56.616390 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 15:35:56.616396 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 15:35:56.616428 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0809 15:35:56.617918 (+ 1490us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 15:35:56.618520 (+ 602us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 15:35:56.618523 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 15:35:56.618524 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 15:35:56.618593 (+ 69us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 15:35:56.619025 (+ 432us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 15:35:56.619027 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 15:35:56.619028 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 15:35:56.619186 (+ 158us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 15:35:56.619264 (+ 78us) server_negotiation.cc:1035] Waiting for connection context
0809 15:35:56.619918 (+ 654us) server_negotiation.cc:299] Negotiation successful
0809 15:35:56.620023 (+ 105us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":68}
I20260809 15:35:56.621368 8183 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:35:56.608423 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.89.254:45901 (local address 127.3.89.194:47781)
0809 15:35:56.608874 (+ 451us) negotiation.cc:107] Waiting for socket to connect
0809 15:35:56.608914 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0809 15:35:56.610113 (+ 1199us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 15:35:56.610580 (+ 467us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 15:35:56.610587 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 15:35:56.610927 (+ 340us) client_negotiation.cc:189] Negotiated authn=SASL
0809 15:35:56.611440 (+ 513us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:35:56.611453 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:35:56.612207 (+ 754us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:35:56.612210 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:35:56.613186 (+ 976us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:35:56.613196 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:35:56.613825 (+ 629us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:35:56.613830 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:35:56.614016 (+ 186us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:35:56.614639 (+ 623us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 15:35:56.614660 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0809 15:35:56.616215 (+ 1555us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 15:35:56.618033 (+ 1818us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 15:35:56.618042 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 15:35:56.618058 (+ 16us) client_negotiation.cc:760] calling sasl_client_step()
0809 15:35:56.618418 (+ 360us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 15:35:56.618678 (+ 260us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 15:35:56.618684 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 15:35:56.618689 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0809 15:35:56.618871 (+ 182us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 15:35:56.619300 (+ 429us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 15:35:56.619309 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 15:35:56.619661 (+ 352us) client_negotiation.cc:769] Sending connection context
0809 15:35:56.619881 (+ 220us) client_negotiation.cc:240] Negotiation successful
0809 15:35:56.620217 (+ 336us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":276,"thread_start_us":132,"threads_started":1}
I20260809 15:35:56.622689 8181 heartbeater.cc:347] Connected to a master server at 127.3.89.254:45901
I20260809 15:35:56.622943 8181 heartbeater.cc:464] Registering TS with master...
I20260809 15:35:56.623528 8181 heartbeater.cc:511] Master 127.3.89.254:45901 requested a full tablet report, sending...
WARNING: no policy specified for kudu/127.3.89.195@KRBTEST.COM; defaulting to no policy
I20260809 15:35:56.624678 7838 ts_manager.cc:194] Registered new tserver with Master: c0c52efb3a8b4a5d9afa1c03c4790182 (127.3.89.194:44803)
I20260809 15:35:56.625277 7838 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.3.89.194@KRBTEST.COM'} at 127.3.89.194:47781
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.3.89.195@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.3.89.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786289732823074-3431-0/krb5kdc/kudu_127.3.89.195.keytab.
Entry for principal kudu/127.3.89.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786289732823074-3431-0/krb5kdc/kudu_127.3.89.195.keytab.
I20260809 15:35:56.642136 3431 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskm314YR/build/debug/bin/kudu
/tmp/dist-test-taskm314YR/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786289732823074-3431-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786289732823074-3431-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786289732823074-3431-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786289732823074-3431-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.3.89.195:0
--local_ip_for_outbound_sockets=127.3.89.195
--webserver_interface=127.3.89.195
--webserver_port=0
--tserver_master_addrs=127.3.89.254:45901
--keytab_file=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786289732823074-3431-0/krb5kdc/kudu_127.3.89.195.keytab
--principal=kudu/127.3.89.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.3.89.212:43745
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--webserver_enabled=0 with env {KRB5CCNAME=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786289732823074-3431-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786289732823074-3431-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786289732823074-3431-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 15:35:56.755885 8186 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 15:35:56.756234 8186 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 15:35:56.756349 8186 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 15:35:56.760506 8186 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 15:35:56.760600 8186 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 15:35:56.760699 8186 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.3.89.195
I20260809 15:35:56.766144 8186 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.3.89.212:43745
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786289732823074-3431-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786289732823074-3431-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786289732823074-3431-0/krb5kdc/kudu_127.3.89.195.keytab
--principal=kudu/127.3.89.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.3.89.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--webserver_enabled=false
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786289732823074-3431-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.3.89.195
--webserver_port=0
--tserver_master_addrs=127.3.89.254:45901
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.8186
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.3.89.195
--log_dir=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786289732823074-3431-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 09 Aug 2026 14:46:55 UTC on ccycloud.quasar-yswkmf.root.comops.site
build id 4576540
I20260809 15:35:56.767242 8186 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 15:35:56.768335 8186 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 15:35:56.778092 8191 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 15:35:56.778153 8194 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 15:35:56.778113 8192 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 15:35:56.778379 8186 server_base.cc:1034] running on GCE node
I20260809 15:35:56.778791 8186 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 15:35:56.779444 8186 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 15:35:56.780599 8186 hybrid_clock.cc:648] HybridClock initialized: now 1786289756780570 us; error 41 us; skew 500 ppm
Aug 09 15:35:56 dist-test-slave-2w3w krb5kdc[7792](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786289756, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.89.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 15:35:56.783289 8186 init.cc:377] Logged in from keytab as kudu/127.3.89.195@KRBTEST.COM (short username kudu)
I20260809 15:35:56.784044 8186 fs_manager.cc:359] Metadata directory not provided
I20260809 15:35:56.784140 8186 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 15:35:56.784376 8186 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 15:35:56.786383 8186 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786289732823074-3431-0/minicluster-data/ts-2/data/instance:
uuid: "05b1c07a4ce24c3bb71e0ef673b85cdd"
format_stamp: "Formatted at 2026-08-09 15:35:56 on dist-test-slave-2w3w"
server_key: "1456df5ff42ee21a9f17d4cc21066646"
server_key_iv: "ee0ceb540246c92e259a32562195c863"
server_key_version: "encryptionkey@0"
I20260809 15:35:56.786950 8186 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786289732823074-3431-0/minicluster-data/ts-2/wal/instance:
uuid: "05b1c07a4ce24c3bb71e0ef673b85cdd"
format_stamp: "Formatted at 2026-08-09 15:35:56 on dist-test-slave-2w3w"
server_key: "1456df5ff42ee21a9f17d4cc21066646"
server_key_iv: "ee0ceb540246c92e259a32562195c863"
server_key_version: "encryptionkey@0"
I20260809 15:35:56.790527 8186 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.002s sys 0.001s
I20260809 15:35:56.792840 8201 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 15:35:56.793968 8186 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260809 15:35:56.794121 8186 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786289732823074-3431-0/minicluster-data/ts-2/data,/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786289732823074-3431-0/minicluster-data/ts-2/wal
uuid: "05b1c07a4ce24c3bb71e0ef673b85cdd"
format_stamp: "Formatted at 2026-08-09 15:35:56 on dist-test-slave-2w3w"
server_key: "1456df5ff42ee21a9f17d4cc21066646"
server_key_iv: "ee0ceb540246c92e259a32562195c863"
server_key_version: "encryptionkey@0"
I20260809 15:35:56.794231 8186 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786289732823074-3431-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786289732823074-3431-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786289732823074-3431-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 15:35:56.806727 8186 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 15:35:56.809584 8186 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 15:35:56.809793 8186 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 15:35:56.810359 8186 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 15:35:56.811244 8186 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 15:35:56.811322 8186 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 15:35:56.811394 8186 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 15:35:56.811437 8186 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 15:35:56.820530 8186 rpc_server.cc:307] RPC server started. Bound to: 127.3.89.195:43427
I20260809 15:35:56.820559 8321 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.3.89.195:43427 every 8 connection(s)
I20260809 15:35:56.821183 8186 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786289732823074-3431-0/minicluster-data/ts-2/data/info.pb
I20260809 15:35:56.829074 3431 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskm314YR/build/debug/bin/kudu as pid 8186
I20260809 15:35:56.829219 3431 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786289732823074-3431-0/minicluster-data/ts-2/wal/instance
I20260809 15:35:56.829514 3431 external_mini_cluster.cc:1428] Setting key 3e7cf575de04c830b53dfee60b2c4c6c
Aug 09 15:35:56 dist-test-slave-2w3w krb5kdc[7792](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786289756, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.89.195@KRBTEST.COM for kudu/127.3.89.254@KRBTEST.COM
I20260809 15:35:56.834937 7881 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:35:56.823010 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.89.195:50447 (local address 127.3.89.254:45901)
0809 15:35:56.823160 (+ 150us) server_negotiation.cc:206] Beginning negotiation
0809 15:35:56.823165 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 15:35:56.824020 (+ 855us) server_negotiation.cc:407] Connection header received
0809 15:35:56.825268 (+ 1248us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 15:35:56.825271 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 15:35:56.825319 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 15:35:56.825412 (+ 93us) server_negotiation.cc:226] Negotiated authn=SASL
0809 15:35:56.826507 (+ 1095us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:35:56.827113 (+ 606us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:35:56.828207 (+ 1094us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:35:56.828424 (+ 217us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:35:56.828507 (+ 83us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:35:56.831495 (+ 2988us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 15:35:56.831500 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 15:35:56.831503 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 15:35:56.831533 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0809 15:35:56.833103 (+ 1570us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 15:35:56.833581 (+ 478us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 15:35:56.833584 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 15:35:56.833586 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 15:35:56.833668 (+ 82us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 15:35:56.834025 (+ 357us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 15:35:56.834028 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 15:35:56.834029 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 15:35:56.834212 (+ 183us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 15:35:56.834297 (+ 85us) server_negotiation.cc:1035] Waiting for connection context
0809 15:35:56.834713 (+ 416us) server_negotiation.cc:299] Negotiation successful
0809 15:35:56.834822 (+ 109us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":47}
I20260809 15:35:56.835808 8325 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:35:56.823416 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.89.254:45901 (local address 127.3.89.195:50447)
0809 15:35:56.823862 (+ 446us) negotiation.cc:107] Waiting for socket to connect
0809 15:35:56.823904 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0809 15:35:56.825074 (+ 1170us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 15:35:56.825545 (+ 471us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 15:35:56.825551 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 15:35:56.825914 (+ 363us) client_negotiation.cc:189] Negotiated authn=SASL
0809 15:35:56.826371 (+ 457us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:35:56.826384 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:35:56.827222 (+ 838us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:35:56.827224 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:35:56.828103 (+ 879us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:35:56.828112 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:35:56.828552 (+ 440us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:35:56.828555 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:35:56.828692 (+ 137us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:35:56.829333 (+ 641us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 15:35:56.829354 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0809 15:35:56.831309 (+ 1955us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 15:35:56.833220 (+ 1911us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 15:35:56.833225 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 15:35:56.833238 (+ 13us) client_negotiation.cc:760] calling sasl_client_step()
0809 15:35:56.833483 (+ 245us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 15:35:56.833780 (+ 297us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 15:35:56.833782 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 15:35:56.833784 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 15:35:56.833910 (+ 126us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 15:35:56.834307 (+ 397us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 15:35:56.834311 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 15:35:56.834516 (+ 205us) client_negotiation.cc:769] Sending connection context
0809 15:35:56.834662 (+ 146us) client_negotiation.cc:240] Negotiation successful
0809 15:35:56.834931 (+ 269us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":265,"thread_start_us":123,"threads_started":1}
I20260809 15:35:56.836899 8323 heartbeater.cc:347] Connected to a master server at 127.3.89.254:45901
I20260809 15:35:56.837109 8323 heartbeater.cc:464] Registering TS with master...
I20260809 15:35:56.837481 8323 heartbeater.cc:511] Master 127.3.89.254:45901 requested a full tablet report, sending...
I20260809 15:35:56.838382 7838 ts_manager.cc:194] Registered new tserver with Master: 05b1c07a4ce24c3bb71e0ef673b85cdd (127.3.89.195:43427)
I20260809 15:35:56.839017 7838 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.3.89.195@KRBTEST.COM'} at 127.3.89.195:50447
I20260809 15:35:56.844107 3431 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260809 15:35:56.853195 7881 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:35:56.845937 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36224 (local address 127.3.89.254:45901)
0809 15:35:56.846052 (+ 115us) server_negotiation.cc:206] Beginning negotiation
0809 15:35:56.846056 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 15:35:56.846207 (+ 151us) server_negotiation.cc:407] Connection header received
0809 15:35:56.846300 (+ 93us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 15:35:56.846302 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 15:35:56.846340 (+ 38us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 15:35:56.846407 (+ 67us) server_negotiation.cc:226] Negotiated authn=SASL
0809 15:35:56.847141 (+ 734us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:35:56.847638 (+ 497us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:35:56.848430 (+ 792us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:35:56.848635 (+ 205us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:35:56.848711 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:35:56.850114 (+ 1403us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 15:35:56.850118 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 15:35:56.850121 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 15:35:56.850149 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0809 15:35:56.851483 (+ 1334us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 15:35:56.851917 (+ 434us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 15:35:56.851921 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 15:35:56.851922 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 15:35:56.852009 (+ 87us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 15:35:56.852292 (+ 283us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 15:35:56.852296 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 15:35:56.852297 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 15:35:56.852530 (+ 233us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 15:35:56.852676 (+ 146us) server_negotiation.cc:1035] Waiting for connection context
0809 15:35:56.852912 (+ 236us) server_negotiation.cc:299] Negotiation successful
0809 15:35:56.853026 (+ 114us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":44}
I20260809 15:35:56.855973 7838 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:36224:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260809 15:35:56.859120 7838 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260809 15:35:56.873436 8334 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:35:56.867519 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.89.195:43427 (local address 127.0.0.1:41550)
0809 15:35:56.867898 (+ 379us) negotiation.cc:107] Waiting for socket to connect
0809 15:35:56.867952 (+ 54us) client_negotiation.cc:174] Beginning negotiation
0809 15:35:56.868090 (+ 138us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 15:35:56.868871 (+ 781us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 15:35:56.868877 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 15:35:56.868900 (+ 23us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 15:35:56.869051 (+ 151us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:35:56.869059 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:35:56.870286 (+ 1227us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:35:56.870290 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:35:56.871746 (+ 1456us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:35:56.871757 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:35:56.872867 (+ 1110us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:35:56.872870 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:35:56.872980 (+ 110us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:35:56.873018 (+ 38us) client_negotiation.cc:769] Sending connection context
0809 15:35:56.873138 (+ 120us) client_negotiation.cc:240] Negotiation successful
0809 15:35:56.873260 (+ 122us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":198,"thread_start_us":123,"threads_started":1}
I20260809 15:35:56.873436 8335 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:35:56.867656 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41550 (local address 127.3.89.195:43427)
0809 15:35:56.868391 (+ 735us) server_negotiation.cc:206] Beginning negotiation
0809 15:35:56.868405 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0809 15:35:56.868431 (+ 26us) server_negotiation.cc:407] Connection header received
0809 15:35:56.868506 (+ 75us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 15:35:56.868512 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 15:35:56.868715 (+ 203us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 15:35:56.868882 (+ 167us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 15:35:56.869198 (+ 316us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:35:56.870147 (+ 949us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:35:56.871887 (+ 1740us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:35:56.872727 (+ 840us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:35:56.872847 (+ 120us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:35:56.873013 (+ 166us) server_negotiation.cc:1035] Waiting for connection context
0809 15:35:56.873173 (+ 160us) server_negotiation.cc:299] Negotiation successful
0809 15:35:56.873268 (+ 95us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":627,"thread_start_us":114,"threads_started":1}
I20260809 15:35:56.873499 8338 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:35:56.868361 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60728 (local address 127.3.89.194:44803)
0809 15:35:56.868937 (+ 576us) server_negotiation.cc:206] Beginning negotiation
0809 15:35:56.868950 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0809 15:35:56.868971 (+ 21us) server_negotiation.cc:407] Connection header received
0809 15:35:56.869044 (+ 73us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 15:35:56.869052 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 15:35:56.869358 (+ 306us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 15:35:56.869475 (+ 117us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 15:35:56.869727 (+ 252us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:35:56.870680 (+ 953us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:35:56.872108 (+ 1428us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:35:56.872865 (+ 757us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:35:56.873097 (+ 232us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:35:56.873229 (+ 132us) server_negotiation.cc:1035] Waiting for connection context
0809 15:35:56.873277 (+ 48us) server_negotiation.cc:299] Negotiation successful
0809 15:35:56.873358 (+ 81us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":477,"thread_start_us":146,"threads_started":1}
I20260809 15:35:56.873494 8336 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:35:56.867805 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.89.194:44803 (local address 127.0.0.1:60728)
0809 15:35:56.868051 (+ 246us) negotiation.cc:107] Waiting for socket to connect
0809 15:35:56.868064 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0809 15:35:56.868133 (+ 69us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 15:35:56.869494 (+ 1361us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 15:35:56.869497 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 15:35:56.869516 (+ 19us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 15:35:56.869615 (+ 99us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:35:56.869620 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:35:56.870852 (+ 1232us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:35:56.870856 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:35:56.871961 (+ 1105us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:35:56.871970 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:35:56.872997 (+ 1027us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:35:56.873001 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:35:56.873157 (+ 156us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:35:56.873179 (+ 22us) client_negotiation.cc:769] Sending connection context
0809 15:35:56.873248 (+ 69us) client_negotiation.cc:240] Negotiation successful
0809 15:35:56.873335 (+ 87us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":191,"mutex_wait_us":30,"thread_start_us":99,"threads_started":1}
I20260809 15:35:56.875399 8337 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:35:56.868322 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.89.193:43777 (local address 127.0.0.1:57206)
0809 15:35:56.869163 (+ 841us) negotiation.cc:107] Waiting for socket to connect
0809 15:35:56.869176 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0809 15:35:56.869254 (+ 78us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 15:35:56.870774 (+ 1520us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 15:35:56.870778 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 15:35:56.870797 (+ 19us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 15:35:56.870898 (+ 101us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:35:56.870907 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:35:56.872232 (+ 1325us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:35:56.872239 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:35:56.873732 (+ 1493us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:35:56.873747 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:35:56.875019 (+ 1272us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:35:56.875022 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:35:56.875107 (+ 85us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:35:56.875131 (+ 24us) client_negotiation.cc:769] Sending connection context
0809 15:35:56.875206 (+ 75us) client_negotiation.cc:240] Negotiation successful
0809 15:35:56.875275 (+ 69us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":770,"mutex_wait_us":556,"thread_start_us":91,"threads_started":1}
I20260809 15:35:56.875814 8339 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:35:56.868518 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57206 (local address 127.3.89.193:43777)
0809 15:35:56.870314 (+ 1796us) server_negotiation.cc:206] Beginning negotiation
0809 15:35:56.870319 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 15:35:56.870337 (+ 18us) server_negotiation.cc:407] Connection header received
0809 15:35:56.870450 (+ 113us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 15:35:56.870464 (+ 14us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 15:35:56.870641 (+ 177us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 15:35:56.870794 (+ 153us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 15:35:56.871087 (+ 293us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:35:56.872102 (+ 1015us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:35:56.873880 (+ 1778us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:35:56.874678 (+ 798us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:35:56.874800 (+ 122us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:35:56.874923 (+ 123us) server_negotiation.cc:1035] Waiting for connection context
0809 15:35:56.875577 (+ 654us) server_negotiation.cc:299] Negotiation successful
0809 15:35:56.875670 (+ 93us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":258,"thread_start_us":82,"threads_started":1}
I20260809 15:35:56.876641 8254 tablet_service.cc:1467] Processing CreateTablet for tablet d098bbf4e0db44649ca6bf07056da3ec (DEFAULT_TABLE table=test-table [id=f7c8283c02914a2dbf31ae4e44a8d5b5]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 15:35:56.876639 8109 tablet_service.cc:1467] Processing CreateTablet for tablet d098bbf4e0db44649ca6bf07056da3ec (DEFAULT_TABLE table=test-table [id=f7c8283c02914a2dbf31ae4e44a8d5b5]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 15:35:56.877583 8109 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d098bbf4e0db44649ca6bf07056da3ec. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 15:35:56.877591 8254 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d098bbf4e0db44649ca6bf07056da3ec. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 15:35:56.884188 8342 tablet_bootstrap.cc:492] T d098bbf4e0db44649ca6bf07056da3ec P c0c52efb3a8b4a5d9afa1c03c4790182: Bootstrap starting.
I20260809 15:35:56.884761 8341 tablet_bootstrap.cc:492] T d098bbf4e0db44649ca6bf07056da3ec P 05b1c07a4ce24c3bb71e0ef673b85cdd: Bootstrap starting.
I20260809 15:35:56.885586 7969 tablet_service.cc:1467] Processing CreateTablet for tablet d098bbf4e0db44649ca6bf07056da3ec (DEFAULT_TABLE table=test-table [id=f7c8283c02914a2dbf31ae4e44a8d5b5]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 15:35:56.886493 7969 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d098bbf4e0db44649ca6bf07056da3ec. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 15:35:56.886524 8342 tablet_bootstrap.cc:654] T d098bbf4e0db44649ca6bf07056da3ec P c0c52efb3a8b4a5d9afa1c03c4790182: Neither blocks nor log segments found. Creating new log.
I20260809 15:35:56.886660 8341 tablet_bootstrap.cc:654] T d098bbf4e0db44649ca6bf07056da3ec P 05b1c07a4ce24c3bb71e0ef673b85cdd: Neither blocks nor log segments found. Creating new log.
I20260809 15:35:56.887972 8341 log.cc:824] T d098bbf4e0db44649ca6bf07056da3ec P 05b1c07a4ce24c3bb71e0ef673b85cdd: Log is configured to *not* fsync() on all Append() calls
I20260809 15:35:56.887972 8342 log.cc:824] T d098bbf4e0db44649ca6bf07056da3ec P c0c52efb3a8b4a5d9afa1c03c4790182: Log is configured to *not* fsync() on all Append() calls
I20260809 15:35:56.890038 8341 tablet_bootstrap.cc:492] T d098bbf4e0db44649ca6bf07056da3ec P 05b1c07a4ce24c3bb71e0ef673b85cdd: No bootstrap required, opened a new log
I20260809 15:35:56.890089 8342 tablet_bootstrap.cc:492] T d098bbf4e0db44649ca6bf07056da3ec P c0c52efb3a8b4a5d9afa1c03c4790182: No bootstrap required, opened a new log
I20260809 15:35:56.890235 8341 ts_tablet_manager.cc:1397] T d098bbf4e0db44649ca6bf07056da3ec P 05b1c07a4ce24c3bb71e0ef673b85cdd: Time spent bootstrapping tablet: real 0.006s user 0.004s sys 0.000s
I20260809 15:35:56.890234 8342 ts_tablet_manager.cc:1397] T d098bbf4e0db44649ca6bf07056da3ec P c0c52efb3a8b4a5d9afa1c03c4790182: Time spent bootstrapping tablet: real 0.006s user 0.000s sys 0.004s
I20260809 15:35:56.892590 8345 tablet_bootstrap.cc:492] T d098bbf4e0db44649ca6bf07056da3ec P 52ce2fe4c3b64218b0a11bc0f07fc6de: Bootstrap starting.
I20260809 15:35:56.894099 8342 raft_consensus.cc:348] T d098bbf4e0db44649ca6bf07056da3ec P c0c52efb3a8b4a5d9afa1c03c4790182 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "05b1c07a4ce24c3bb71e0ef673b85cdd" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 43427 } } peers { permanent_uuid: "52ce2fe4c3b64218b0a11bc0f07fc6de" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 43777 } } peers { permanent_uuid: "c0c52efb3a8b4a5d9afa1c03c4790182" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 44803 } }
I20260809 15:35:56.894151 8341 raft_consensus.cc:348] T d098bbf4e0db44649ca6bf07056da3ec P 05b1c07a4ce24c3bb71e0ef673b85cdd [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "05b1c07a4ce24c3bb71e0ef673b85cdd" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 43427 } } peers { permanent_uuid: "52ce2fe4c3b64218b0a11bc0f07fc6de" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 43777 } } peers { permanent_uuid: "c0c52efb3a8b4a5d9afa1c03c4790182" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 44803 } }
I20260809 15:35:56.894400 8342 raft_consensus.cc:374] T d098bbf4e0db44649ca6bf07056da3ec P c0c52efb3a8b4a5d9afa1c03c4790182 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 15:35:56.894400 8341 raft_consensus.cc:374] T d098bbf4e0db44649ca6bf07056da3ec P 05b1c07a4ce24c3bb71e0ef673b85cdd [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 15:35:56.894461 8341 raft_consensus.cc:729] T d098bbf4e0db44649ca6bf07056da3ec P 05b1c07a4ce24c3bb71e0ef673b85cdd [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 05b1c07a4ce24c3bb71e0ef673b85cdd, State: Initialized, Role: FOLLOWER
I20260809 15:35:56.894481 8345 tablet_bootstrap.cc:654] T d098bbf4e0db44649ca6bf07056da3ec P 52ce2fe4c3b64218b0a11bc0f07fc6de: Neither blocks nor log segments found. Creating new log.
I20260809 15:35:56.894503 8342 raft_consensus.cc:729] T d098bbf4e0db44649ca6bf07056da3ec P c0c52efb3a8b4a5d9afa1c03c4790182 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c0c52efb3a8b4a5d9afa1c03c4790182, State: Initialized, Role: FOLLOWER
I20260809 15:35:56.894986 8341 consensus_queue.cc:260] T d098bbf4e0db44649ca6bf07056da3ec P 05b1c07a4ce24c3bb71e0ef673b85cdd [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "05b1c07a4ce24c3bb71e0ef673b85cdd" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 43427 } } peers { permanent_uuid: "52ce2fe4c3b64218b0a11bc0f07fc6de" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 43777 } } peers { permanent_uuid: "c0c52efb3a8b4a5d9afa1c03c4790182" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 44803 } }
I20260809 15:35:56.894986 8342 consensus_queue.cc:260] T d098bbf4e0db44649ca6bf07056da3ec P c0c52efb3a8b4a5d9afa1c03c4790182 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "05b1c07a4ce24c3bb71e0ef673b85cdd" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 43427 } } peers { permanent_uuid: "52ce2fe4c3b64218b0a11bc0f07fc6de" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 43777 } } peers { permanent_uuid: "c0c52efb3a8b4a5d9afa1c03c4790182" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 44803 } }
I20260809 15:35:56.895865 8323 heartbeater.cc:503] Master 127.3.89.254:45901 was elected leader, sending a full tablet report...
I20260809 15:35:56.895974 8342 ts_tablet_manager.cc:1428] T d098bbf4e0db44649ca6bf07056da3ec P c0c52efb3a8b4a5d9afa1c03c4790182: Time spent starting tablet: real 0.006s user 0.006s sys 0.000s
I20260809 15:35:56.895974 8341 ts_tablet_manager.cc:1428] T d098bbf4e0db44649ca6bf07056da3ec P 05b1c07a4ce24c3bb71e0ef673b85cdd: Time spent starting tablet: real 0.006s user 0.006s sys 0.000s
I20260809 15:35:56.896540 8181 heartbeater.cc:503] Master 127.3.89.254:45901 was elected leader, sending a full tablet report...
I20260809 15:35:56.896589 8345 log.cc:824] T d098bbf4e0db44649ca6bf07056da3ec P 52ce2fe4c3b64218b0a11bc0f07fc6de: Log is configured to *not* fsync() on all Append() calls
I20260809 15:35:56.898725 8345 tablet_bootstrap.cc:492] T d098bbf4e0db44649ca6bf07056da3ec P 52ce2fe4c3b64218b0a11bc0f07fc6de: No bootstrap required, opened a new log
I20260809 15:35:56.898926 8345 ts_tablet_manager.cc:1397] T d098bbf4e0db44649ca6bf07056da3ec P 52ce2fe4c3b64218b0a11bc0f07fc6de: Time spent bootstrapping tablet: real 0.006s user 0.004s sys 0.000s
I20260809 15:35:56.902112 8345 raft_consensus.cc:348] T d098bbf4e0db44649ca6bf07056da3ec P 52ce2fe4c3b64218b0a11bc0f07fc6de [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "05b1c07a4ce24c3bb71e0ef673b85cdd" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 43427 } } peers { permanent_uuid: "52ce2fe4c3b64218b0a11bc0f07fc6de" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 43777 } } peers { permanent_uuid: "c0c52efb3a8b4a5d9afa1c03c4790182" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 44803 } }
I20260809 15:35:56.902326 8345 raft_consensus.cc:374] T d098bbf4e0db44649ca6bf07056da3ec P 52ce2fe4c3b64218b0a11bc0f07fc6de [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 15:35:56.902361 8345 raft_consensus.cc:729] T d098bbf4e0db44649ca6bf07056da3ec P 52ce2fe4c3b64218b0a11bc0f07fc6de [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 52ce2fe4c3b64218b0a11bc0f07fc6de, State: Initialized, Role: FOLLOWER
I20260809 15:35:56.902738 8345 consensus_queue.cc:260] T d098bbf4e0db44649ca6bf07056da3ec P 52ce2fe4c3b64218b0a11bc0f07fc6de [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "05b1c07a4ce24c3bb71e0ef673b85cdd" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 43427 } } peers { permanent_uuid: "52ce2fe4c3b64218b0a11bc0f07fc6de" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 43777 } } peers { permanent_uuid: "c0c52efb3a8b4a5d9afa1c03c4790182" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 44803 } }
I20260809 15:35:56.903369 8038 heartbeater.cc:503] Master 127.3.89.254:45901 was elected leader, sending a full tablet report...
I20260809 15:35:56.903713 8345 ts_tablet_manager.cc:1428] T d098bbf4e0db44649ca6bf07056da3ec P 52ce2fe4c3b64218b0a11bc0f07fc6de: Time spent starting tablet: real 0.005s user 0.004s sys 0.000s
W20260809 15:35:57.072063 8324 tablet.cc:2369] T d098bbf4e0db44649ca6bf07056da3ec P 05b1c07a4ce24c3bb71e0ef673b85cdd: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260809 15:35:57.107422 8182 tablet.cc:2369] T d098bbf4e0db44649ca6bf07056da3ec P c0c52efb3a8b4a5d9afa1c03c4790182: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260809 15:35:57.138720 8039 tablet.cc:2369] T d098bbf4e0db44649ca6bf07056da3ec P 52ce2fe4c3b64218b0a11bc0f07fc6de: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260809 15:35:57.218832 8349 raft_consensus.cc:482] T d098bbf4e0db44649ca6bf07056da3ec P 52ce2fe4c3b64218b0a11bc0f07fc6de [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 15:35:57.219063 8349 raft_consensus.cc:504] T d098bbf4e0db44649ca6bf07056da3ec P 52ce2fe4c3b64218b0a11bc0f07fc6de [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "05b1c07a4ce24c3bb71e0ef673b85cdd" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 43427 } } peers { permanent_uuid: "52ce2fe4c3b64218b0a11bc0f07fc6de" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 43777 } } peers { permanent_uuid: "c0c52efb3a8b4a5d9afa1c03c4790182" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 44803 } }
I20260809 15:35:57.220270 8349 leader_election.cc:288] T d098bbf4e0db44649ca6bf07056da3ec P 52ce2fe4c3b64218b0a11bc0f07fc6de [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 05b1c07a4ce24c3bb71e0ef673b85cdd (127.3.89.195:43427), c0c52efb3a8b4a5d9afa1c03c4790182 (127.3.89.194:44803)
I20260809 15:35:57.224534 8335 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:35:57.220657 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.89.193:34925 (local address 127.3.89.195:43427)
0809 15:35:57.220846 (+ 189us) server_negotiation.cc:206] Beginning negotiation
0809 15:35:57.220853 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 15:35:57.221218 (+ 365us) server_negotiation.cc:407] Connection header received
0809 15:35:57.221377 (+ 159us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 15:35:57.221380 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 15:35:57.221445 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 15:35:57.221532 (+ 87us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 15:35:57.221837 (+ 305us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:35:57.222440 (+ 603us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:35:57.223584 (+ 1144us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:35:57.223985 (+ 401us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:35:57.224117 (+ 132us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:35:57.224169 (+ 52us) server_negotiation.cc:1035] Waiting for connection context
0809 15:35:57.224343 (+ 174us) server_negotiation.cc:299] Negotiation successful
0809 15:35:57.224409 (+ 66us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":103}
I20260809 15:35:57.224552 8351 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:35:57.220528 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.89.195:43427 (local address 127.3.89.193:34925)
0809 15:35:57.221088 (+ 560us) negotiation.cc:107] Waiting for socket to connect
0809 15:35:57.221122 (+ 34us) client_negotiation.cc:174] Beginning negotiation
0809 15:35:57.221265 (+ 143us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 15:35:57.221544 (+ 279us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 15:35:57.221547 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 15:35:57.221563 (+ 16us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 15:35:57.221716 (+ 153us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:35:57.221722 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:35:57.222547 (+ 825us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:35:57.222551 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:35:57.223466 (+ 915us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:35:57.223475 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:35:57.224122 (+ 647us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:35:57.224126 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:35:57.224212 (+ 86us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:35:57.224238 (+ 26us) client_negotiation.cc:769] Sending connection context
0809 15:35:57.224328 (+ 90us) client_negotiation.cc:240] Negotiation successful
0809 15:35:57.224403 (+ 75us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":492,"mutex_wait_us":28,"thread_start_us":153,"threads_started":1}
I20260809 15:35:57.224802 8350 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:35:57.220521 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.89.194:44803 (local address 127.3.89.193:36415)
0809 15:35:57.220973 (+ 452us) negotiation.cc:107] Waiting for socket to connect
0809 15:35:57.220987 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0809 15:35:57.221190 (+ 203us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 15:35:57.221493 (+ 303us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 15:35:57.221497 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 15:35:57.221514 (+ 17us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 15:35:57.221687 (+ 173us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:35:57.221694 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:35:57.222616 (+ 922us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:35:57.222622 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:35:57.223528 (+ 906us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:35:57.223538 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:35:57.224310 (+ 772us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:35:57.224314 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:35:57.224448 (+ 134us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:35:57.224477 (+ 29us) client_negotiation.cc:769] Sending connection context
0809 15:35:57.224573 (+ 96us) client_negotiation.cc:240] Negotiation successful
0809 15:35:57.224645 (+ 72us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":373,"mutex_wait_us":8,"thread_start_us":131,"threads_started":1}
I20260809 15:35:57.224792 8338 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:35:57.220711 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.89.193:36415 (local address 127.3.89.194:44803)
0809 15:35:57.220988 (+ 277us) server_negotiation.cc:206] Beginning negotiation
0809 15:35:57.220994 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 15:35:57.221069 (+ 75us) server_negotiation.cc:407] Connection header received
0809 15:35:57.221327 (+ 258us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 15:35:57.221336 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 15:35:57.221383 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 15:35:57.221481 (+ 98us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 15:35:57.221818 (+ 337us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:35:57.222481 (+ 663us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:35:57.223667 (+ 1186us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:35:57.224162 (+ 495us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:35:57.224307 (+ 145us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:35:57.224368 (+ 61us) server_negotiation.cc:1035] Waiting for connection context
0809 15:35:57.224589 (+ 221us) server_negotiation.cc:299] Negotiation successful
0809 15:35:57.224659 (+ 70us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":179}
I20260809 15:35:57.225535 8274 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d098bbf4e0db44649ca6bf07056da3ec" candidate_uuid: "52ce2fe4c3b64218b0a11bc0f07fc6de" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "05b1c07a4ce24c3bb71e0ef673b85cdd" is_pre_election: true
I20260809 15:35:57.225535 8131 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d098bbf4e0db44649ca6bf07056da3ec" candidate_uuid: "52ce2fe4c3b64218b0a11bc0f07fc6de" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c0c52efb3a8b4a5d9afa1c03c4790182" is_pre_election: true
I20260809 15:35:57.225829 8274 raft_consensus.cc:2454] T d098bbf4e0db44649ca6bf07056da3ec P 05b1c07a4ce24c3bb71e0ef673b85cdd [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 52ce2fe4c3b64218b0a11bc0f07fc6de in term 0.
I20260809 15:35:57.225857 8131 raft_consensus.cc:2454] T d098bbf4e0db44649ca6bf07056da3ec P c0c52efb3a8b4a5d9afa1c03c4790182 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 52ce2fe4c3b64218b0a11bc0f07fc6de in term 0.
I20260809 15:35:57.226382 7922 leader_election.cc:302] T d098bbf4e0db44649ca6bf07056da3ec P 52ce2fe4c3b64218b0a11bc0f07fc6de [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 52ce2fe4c3b64218b0a11bc0f07fc6de, c0c52efb3a8b4a5d9afa1c03c4790182; no voters:
I20260809 15:35:57.226647 8349 raft_consensus.cc:2781] T d098bbf4e0db44649ca6bf07056da3ec P 52ce2fe4c3b64218b0a11bc0f07fc6de [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260809 15:35:57.226732 8349 raft_consensus.cc:482] T d098bbf4e0db44649ca6bf07056da3ec P 52ce2fe4c3b64218b0a11bc0f07fc6de [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 15:35:57.226778 8349 raft_consensus.cc:3037] T d098bbf4e0db44649ca6bf07056da3ec P 52ce2fe4c3b64218b0a11bc0f07fc6de [term 0 FOLLOWER]: Advancing to term 1
I20260809 15:35:57.227934 8349 raft_consensus.cc:504] T d098bbf4e0db44649ca6bf07056da3ec P 52ce2fe4c3b64218b0a11bc0f07fc6de [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "05b1c07a4ce24c3bb71e0ef673b85cdd" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 43427 } } peers { permanent_uuid: "52ce2fe4c3b64218b0a11bc0f07fc6de" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 43777 } } peers { permanent_uuid: "c0c52efb3a8b4a5d9afa1c03c4790182" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 44803 } }
I20260809 15:35:57.228305 8349 leader_election.cc:288] T d098bbf4e0db44649ca6bf07056da3ec P 52ce2fe4c3b64218b0a11bc0f07fc6de [CANDIDATE]: Term 1 election: Requested vote from peers 05b1c07a4ce24c3bb71e0ef673b85cdd (127.3.89.195:43427), c0c52efb3a8b4a5d9afa1c03c4790182 (127.3.89.194:44803)
I20260809 15:35:57.228700 8131 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d098bbf4e0db44649ca6bf07056da3ec" candidate_uuid: "52ce2fe4c3b64218b0a11bc0f07fc6de" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c0c52efb3a8b4a5d9afa1c03c4790182"
I20260809 15:35:57.228719 8274 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d098bbf4e0db44649ca6bf07056da3ec" candidate_uuid: "52ce2fe4c3b64218b0a11bc0f07fc6de" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "05b1c07a4ce24c3bb71e0ef673b85cdd"
I20260809 15:35:57.228857 8131 raft_consensus.cc:3037] T d098bbf4e0db44649ca6bf07056da3ec P c0c52efb3a8b4a5d9afa1c03c4790182 [term 0 FOLLOWER]: Advancing to term 1
I20260809 15:35:57.228859 8274 raft_consensus.cc:3037] T d098bbf4e0db44649ca6bf07056da3ec P 05b1c07a4ce24c3bb71e0ef673b85cdd [term 0 FOLLOWER]: Advancing to term 1
I20260809 15:35:57.229897 8274 raft_consensus.cc:2454] T d098bbf4e0db44649ca6bf07056da3ec P 05b1c07a4ce24c3bb71e0ef673b85cdd [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 52ce2fe4c3b64218b0a11bc0f07fc6de in term 1.
I20260809 15:35:57.230094 8131 raft_consensus.cc:2454] T d098bbf4e0db44649ca6bf07056da3ec P c0c52efb3a8b4a5d9afa1c03c4790182 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 52ce2fe4c3b64218b0a11bc0f07fc6de in term 1.
I20260809 15:35:57.230225 7920 leader_election.cc:302] T d098bbf4e0db44649ca6bf07056da3ec P 52ce2fe4c3b64218b0a11bc0f07fc6de [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 05b1c07a4ce24c3bb71e0ef673b85cdd, 52ce2fe4c3b64218b0a11bc0f07fc6de; no voters:
I20260809 15:35:57.230405 8349 raft_consensus.cc:2781] T d098bbf4e0db44649ca6bf07056da3ec P 52ce2fe4c3b64218b0a11bc0f07fc6de [term 1 FOLLOWER]: Leader election won for term 1
I20260809 15:35:57.230618 8349 raft_consensus.cc:686] T d098bbf4e0db44649ca6bf07056da3ec P 52ce2fe4c3b64218b0a11bc0f07fc6de [term 1 LEADER]: Becoming Leader. State: Replica: 52ce2fe4c3b64218b0a11bc0f07fc6de, State: Running, Role: LEADER
I20260809 15:35:57.230945 8349 consensus_queue.cc:237] T d098bbf4e0db44649ca6bf07056da3ec P 52ce2fe4c3b64218b0a11bc0f07fc6de [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "05b1c07a4ce24c3bb71e0ef673b85cdd" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 43427 } } peers { permanent_uuid: "52ce2fe4c3b64218b0a11bc0f07fc6de" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 43777 } } peers { permanent_uuid: "c0c52efb3a8b4a5d9afa1c03c4790182" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 44803 } }
I20260809 15:35:57.234155 7838 catalog_manager.cc:5495] T d098bbf4e0db44649ca6bf07056da3ec P 52ce2fe4c3b64218b0a11bc0f07fc6de reported cstate change: term changed from 0 to 1, leader changed from <none> to 52ce2fe4c3b64218b0a11bc0f07fc6de (127.3.89.193). New cstate: current_term: 1 leader_uuid: "52ce2fe4c3b64218b0a11bc0f07fc6de" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "05b1c07a4ce24c3bb71e0ef673b85cdd" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 43427 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "52ce2fe4c3b64218b0a11bc0f07fc6de" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 43777 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "c0c52efb3a8b4a5d9afa1c03c4790182" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 44803 } health_report { overall_health: UNKNOWN } } }
I20260809 15:35:57.323792 8339 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:35:57.320013 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57216 (local address 127.3.89.193:43777)
0809 15:35:57.320163 (+ 150us) server_negotiation.cc:206] Beginning negotiation
0809 15:35:57.320170 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 15:35:57.320188 (+ 18us) server_negotiation.cc:407] Connection header received
0809 15:35:57.320307 (+ 119us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 15:35:57.320314 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 15:35:57.320418 (+ 104us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 15:35:57.320532 (+ 114us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 15:35:57.320825 (+ 293us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:35:57.321388 (+ 563us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:35:57.322270 (+ 882us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:35:57.322500 (+ 230us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:35:57.322607 (+ 107us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:35:57.322940 (+ 333us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 15:35:57.323417 (+ 477us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 15:35:57.323528 (+ 111us) server_negotiation.cc:1035] Waiting for connection context
0809 15:35:57.323624 (+ 96us) server_negotiation.cc:299] Negotiation successful
0809 15:35:57.323676 (+ 52us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":60}
I20260809 15:35:57.329607 8274 raft_consensus.cc:1261] T d098bbf4e0db44649ca6bf07056da3ec P 05b1c07a4ce24c3bb71e0ef673b85cdd [term 1 FOLLOWER]: Refusing update from remote peer 52ce2fe4c3b64218b0a11bc0f07fc6de: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 15:35:57.329641 8131 raft_consensus.cc:1261] T d098bbf4e0db44649ca6bf07056da3ec P c0c52efb3a8b4a5d9afa1c03c4790182 [term 1 FOLLOWER]: Refusing update from remote peer 52ce2fe4c3b64218b0a11bc0f07fc6de: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 15:35:57.330262 8349 consensus_queue.cc:1035] T d098bbf4e0db44649ca6bf07056da3ec P 52ce2fe4c3b64218b0a11bc0f07fc6de [LEADER]: Connected to new peer: Peer: permanent_uuid: "05b1c07a4ce24c3bb71e0ef673b85cdd" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 43427 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 15:35:57.330478 8352 consensus_queue.cc:1035] T d098bbf4e0db44649ca6bf07056da3ec P 52ce2fe4c3b64218b0a11bc0f07fc6de [LEADER]: Connected to new peer: Peer: permanent_uuid: "c0c52efb3a8b4a5d9afa1c03c4790182" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 44803 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 15:35:57.337913 8356 mvcc.cc:205] Tried to move back new op lower bound from 7316642846014713856 to 7316642845620584448. Current Snapshot: MvccSnapshot[applied={T|T < 7316642846014713856}]
I20260809 15:35:57.346386 7838 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:36224:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 15:35:57.346580 7838 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:36224:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 15:35:57.349095 7838 catalog_manager.cc:5782] T 00000000000000000000000000000000 P e41b0a7c3aa14466942eac2d1058406a: Sending DeleteTablet for 3 replicas of tablet d098bbf4e0db44649ca6bf07056da3ec
I20260809 15:35:57.349936 7969 tablet_service.cc:1514] Processing DeleteTablet for tablet d098bbf4e0db44649ca6bf07056da3ec with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 15:35:57 UTC) from {username='kudu', principal='kudu/127.3.89.254@KRBTEST.COM'} at 127.0.0.1:57206
I20260809 15:35:57.349928 8109 tablet_service.cc:1514] Processing DeleteTablet for tablet d098bbf4e0db44649ca6bf07056da3ec with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 15:35:57 UTC) from {username='kudu', principal='kudu/127.3.89.254@KRBTEST.COM'} at 127.0.0.1:60728
I20260809 15:35:57.350500 8368 tablet_replica.cc:330] T d098bbf4e0db44649ca6bf07056da3ec P 52ce2fe4c3b64218b0a11bc0f07fc6de: stopping tablet replica
I20260809 15:35:57.350795 8368 raft_consensus.cc:2229] T d098bbf4e0db44649ca6bf07056da3ec P 52ce2fe4c3b64218b0a11bc0f07fc6de [term 1 LEADER]: Raft consensus shutting down.
I20260809 15:35:57.350771 8254 tablet_service.cc:1514] Processing DeleteTablet for tablet d098bbf4e0db44649ca6bf07056da3ec with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 15:35:57 UTC) from {username='kudu', principal='kudu/127.3.89.254@KRBTEST.COM'} at 127.0.0.1:41550
I20260809 15:35:57.351074 3431 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskm314YR/build/debug/bin/kudu with pid 7904
I20260809 15:35:57.351148 8368 raft_consensus.cc:2258] T d098bbf4e0db44649ca6bf07056da3ec P 52ce2fe4c3b64218b0a11bc0f07fc6de [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 15:35:57.351465 8367 tablet_replica.cc:330] T d098bbf4e0db44649ca6bf07056da3ec P c0c52efb3a8b4a5d9afa1c03c4790182: stopping tablet replica
I20260809 15:35:57.351763 8367 raft_consensus.cc:2229] T d098bbf4e0db44649ca6bf07056da3ec P c0c52efb3a8b4a5d9afa1c03c4790182 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 15:35:57.352000 8367 raft_consensus.cc:2258] T d098bbf4e0db44649ca6bf07056da3ec P c0c52efb3a8b4a5d9afa1c03c4790182 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 15:35:57.353317 8367 ts_tablet_manager.cc:1905] T d098bbf4e0db44649ca6bf07056da3ec P c0c52efb3a8b4a5d9afa1c03c4790182: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 15:35:57.355945 8367 ts_tablet_manager.cc:1918] T d098bbf4e0db44649ca6bf07056da3ec P c0c52efb3a8b4a5d9afa1c03c4790182: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 15:35:57.356057 8367 log.cc:1196] T d098bbf4e0db44649ca6bf07056da3ec P c0c52efb3a8b4a5d9afa1c03c4790182: Deleting WAL directory at /tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786289732823074-3431-0/minicluster-data/ts-1/wal/wals/d098bbf4e0db44649ca6bf07056da3ec
I20260809 15:35:57.356369 8367 ts_tablet_manager.cc:1939] T d098bbf4e0db44649ca6bf07056da3ec P c0c52efb3a8b4a5d9afa1c03c4790182: Deleting consensus metadata
I20260809 15:35:57.357638 7824 catalog_manager.cc:4841] TS c0c52efb3a8b4a5d9afa1c03c4790182 (127.3.89.194:44803): tablet d098bbf4e0db44649ca6bf07056da3ec (table test-table [id=f7c8283c02914a2dbf31ae4e44a8d5b5]) successfully deleted
I20260809 15:35:57.359190 8369 tablet_replica.cc:330] T d098bbf4e0db44649ca6bf07056da3ec P 05b1c07a4ce24c3bb71e0ef673b85cdd: stopping tablet replica
I20260809 15:35:57.359469 8369 raft_consensus.cc:2229] T d098bbf4e0db44649ca6bf07056da3ec P 05b1c07a4ce24c3bb71e0ef673b85cdd [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 15:35:57.359756 8369 raft_consensus.cc:2258] T d098bbf4e0db44649ca6bf07056da3ec P 05b1c07a4ce24c3bb71e0ef673b85cdd [term 1 FOLLOWER]: Raft consensus is shut down!
W20260809 15:35:57.360880 7822 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.3.89.193:43777 (error 108)
I20260809 15:35:57.361027 3431 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskm314YR/build/debug/bin/kudu with pid 8043
I20260809 15:35:57.362763 8369 ts_tablet_manager.cc:1905] T d098bbf4e0db44649ca6bf07056da3ec P 05b1c07a4ce24c3bb71e0ef673b85cdd: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 15:35:57.366357 8369 ts_tablet_manager.cc:1918] T d098bbf4e0db44649ca6bf07056da3ec P 05b1c07a4ce24c3bb71e0ef673b85cdd: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 15:35:57.366465 8369 log.cc:1196] T d098bbf4e0db44649ca6bf07056da3ec P 05b1c07a4ce24c3bb71e0ef673b85cdd: Deleting WAL directory at /tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786289732823074-3431-0/minicluster-data/ts-2/wal/wals/d098bbf4e0db44649ca6bf07056da3ec
I20260809 15:35:57.366813 8369 ts_tablet_manager.cc:1939] T d098bbf4e0db44649ca6bf07056da3ec P 05b1c07a4ce24c3bb71e0ef673b85cdd: Deleting consensus metadata
I20260809 15:35:57.367714 8337 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:35:57.367441 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.89.193:43777 (local address 127.0.0.1:57224)
0809 15:35:57.367574 (+ 133us) negotiation.cc:107] Waiting for socket to connect
0809 15:35:57.367641 (+ 67us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.89.193:43777: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":48}
W20260809 15:35:57.367918 7822 catalog_manager.cc:4584] TS 52ce2fe4c3b64218b0a11bc0f07fc6de (127.3.89.193:43777): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet d098bbf4e0db44649ca6bf07056da3ec: Network error: Client connection negotiation failed: client connection to 127.3.89.193:43777: connect: Connection refused (error 111)
I20260809 15:35:57.368146 7822 catalog_manager.cc:4841] TS 05b1c07a4ce24c3bb71e0ef673b85cdd (127.3.89.195:43427): tablet d098bbf4e0db44649ca6bf07056da3ec (table test-table [id=f7c8283c02914a2dbf31ae4e44a8d5b5]) successfully deleted
I20260809 15:35:57.369153 3431 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskm314YR/build/debug/bin/kudu with pid 8186
I20260809 15:35:57.375938 3431 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskm314YR/build/debug/bin/kudu with pid 7806
2026-08-09T15:35:57Z chronyd exiting
[ OK ] SecurityITest.TestDisableWebUI (2674 ms)
[ RUN ] SecurityITest.TestDisableAuthenticationEncryption
2026-08-09T15:35:57Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T15:35:57Z Disabled control of system clock
I20260809 15:35:57.408818 3431 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskm314YR/build/debug/bin/kudu
/tmp/dist-test-taskm314YR/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786289732823074-3431-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786289732823074-3431-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786289732823074-3431-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786289732823074-3431-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.3.89.254:38991
--webserver_interface=127.3.89.254
--webserver_port=0
--builtin_ntp_servers=127.3.89.212:44115
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.3.89.254:38991
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_authentication=disabled
--rpc_encryption=disabled with env {}
W20260809 15:35:57.518524 8374 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 15:35:57.518803 8374 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 15:35:57.518839 8374 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 15:35:57.522826 8374 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 15:35:57.522888 8374 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 15:35:57.522904 8374 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 15:35:57.522921 8374 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 15:35:57.522936 8374 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 15:35:57.527699 8374 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.3.89.212:44115
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786289732823074-3431-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786289732823074-3431-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.3.89.254:38991
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.3.89.254:38991
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786289732823074-3431-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.3.89.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.8374
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786289732823074-3431-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 09 Aug 2026 14:46:55 UTC on ccycloud.quasar-yswkmf.root.comops.site
build id 4576540
I20260809 15:35:57.528693 8374 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 15:35:57.529983 8374 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 15:35:57.538494 8380 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 15:35:57.538548 8379 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 15:35:57.538566 8374 server_base.cc:1034] running on GCE node
W20260809 15:35:57.538764 8382 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 15:35:57.539343 8374 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 15:35:57.540349 8374 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 15:35:57.541510 8374 hybrid_clock.cc:648] HybridClock initialized: now 1786289757541492 us; error 29 us; skew 500 ppm
I20260809 15:35:57.543478 8374 webserver.cc:460] Webserver started at http://127.3.89.254:36267/ using document root <none> and password file <none>
I20260809 15:35:57.544059 8374 fs_manager.cc:359] Metadata directory not provided
I20260809 15:35:57.544123 8374 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 15:35:57.544368 8374 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 15:35:57.546396 8374 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786289732823074-3431-0/minicluster-data/master-0/data/instance:
uuid: "da58c50a461a416a83f1a0e86f316b77"
format_stamp: "Formatted at 2026-08-09 15:35:57 on dist-test-slave-2w3w"
server_key: "7294ba5839e168c75e0e9ad1df764e73"
server_key_iv: "d5d843861a262a04ed261898bfadfb3b"
server_key_version: "encryptionkey@0"
I20260809 15:35:57.546994 8374 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786289732823074-3431-0/minicluster-data/master-0/wal/instance:
uuid: "da58c50a461a416a83f1a0e86f316b77"
format_stamp: "Formatted at 2026-08-09 15:35:57 on dist-test-slave-2w3w"
server_key: "7294ba5839e168c75e0e9ad1df764e73"
server_key_iv: "d5d843861a262a04ed261898bfadfb3b"
server_key_version: "encryptionkey@0"
I20260809 15:35:57.550560 8374 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.003s sys 0.000s
I20260809 15:35:57.553092 8388 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 15:35:57.554337 8374 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.000s sys 0.002s
I20260809 15:35:57.554486 8374 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786289732823074-3431-0/minicluster-data/master-0/data,/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786289732823074-3431-0/minicluster-data/master-0/wal
uuid: "da58c50a461a416a83f1a0e86f316b77"
format_stamp: "Formatted at 2026-08-09 15:35:57 on dist-test-slave-2w3w"
server_key: "7294ba5839e168c75e0e9ad1df764e73"
server_key_iv: "d5d843861a262a04ed261898bfadfb3b"
server_key_version: "encryptionkey@0"
I20260809 15:35:57.554601 8374 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786289732823074-3431-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786289732823074-3431-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786289732823074-3431-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 15:35:57.559253 8374 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 15:35:57.559870 8374 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 15:35:57.560060 8374 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 15:35:57.566784 8374 rpc_server.cc:307] RPC server started. Bound to: 127.3.89.254:38991
I20260809 15:35:57.566790 8441 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.3.89.254:38991 every 8 connection(s)
I20260809 15:35:57.567700 8374 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786289732823074-3431-0/minicluster-data/master-0/data/info.pb
I20260809 15:35:57.571368 8442 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 15:35:57.574206 3431 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskm314YR/build/debug/bin/kudu as pid 8374
I20260809 15:35:57.574354 3431 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786289732823074-3431-0/minicluster-data/master-0/wal/instance
I20260809 15:35:57.574630 3431 external_mini_cluster.cc:1428] Setting key 58be907213cb42ed7424b0fbf55c6459
I20260809 15:35:57.577144 8442 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P da58c50a461a416a83f1a0e86f316b77: Bootstrap starting.
I20260809 15:35:57.579810 8442 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P da58c50a461a416a83f1a0e86f316b77: Neither blocks nor log segments found. Creating new log.
I20260809 15:35:57.580358 8445 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:35:57.576090 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51100 (local address 127.3.89.254:38991)
0809 15:35:57.576708 (+ 618us) server_negotiation.cc:206] Beginning negotiation
0809 15:35:57.576723 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0809 15:35:57.576760 (+ 37us) server_negotiation.cc:407] Connection header received
0809 15:35:57.577469 (+ 709us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 15:35:57.577487 (+ 18us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 15:35:57.577830 (+ 343us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 15:35:57.578102 (+ 272us) server_negotiation.cc:226] Negotiated authn=SASL
0809 15:35:57.579020 (+ 918us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 15:35:57.579025 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 15:35:57.579036 (+ 11us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 15:35:57.579049 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0809 15:35:57.579106 (+ 57us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 15:35:57.579122 (+ 16us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 15:35:57.579221 (+ 99us) server_negotiation.cc:1035] Waiting for connection context
0809 15:35:57.579377 (+ 156us) server_negotiation.cc:299] Negotiation successful
0809 15:35:57.579642 (+ 265us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":275,"thread_start_us":148,"threads_started":1}
I20260809 15:35:57.580629 8442 log.cc:824] T 00000000000000000000000000000000 P da58c50a461a416a83f1a0e86f316b77: Log is configured to *not* fsync() on all Append() calls
I20260809 15:35:57.582510 8442 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P da58c50a461a416a83f1a0e86f316b77: No bootstrap required, opened a new log
I20260809 15:35:57.585973 8442 raft_consensus.cc:348] T 00000000000000000000000000000000 P da58c50a461a416a83f1a0e86f316b77 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "da58c50a461a416a83f1a0e86f316b77" member_type: VOTER last_known_addr { host: "127.3.89.254" port: 38991 } }
I20260809 15:35:57.586228 8442 raft_consensus.cc:374] T 00000000000000000000000000000000 P da58c50a461a416a83f1a0e86f316b77 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 15:35:57.586289 8442 raft_consensus.cc:729] T 00000000000000000000000000000000 P da58c50a461a416a83f1a0e86f316b77 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: da58c50a461a416a83f1a0e86f316b77, State: Initialized, Role: FOLLOWER
I20260809 15:35:57.586737 8442 consensus_queue.cc:260] T 00000000000000000000000000000000 P da58c50a461a416a83f1a0e86f316b77 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "da58c50a461a416a83f1a0e86f316b77" member_type: VOTER last_known_addr { host: "127.3.89.254" port: 38991 } }
I20260809 15:35:57.586884 8442 raft_consensus.cc:388] T 00000000000000000000000000000000 P da58c50a461a416a83f1a0e86f316b77 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 15:35:57.586951 8442 raft_consensus.cc:482] T 00000000000000000000000000000000 P da58c50a461a416a83f1a0e86f316b77 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 15:35:57.587096 8442 raft_consensus.cc:3037] T 00000000000000000000000000000000 P da58c50a461a416a83f1a0e86f316b77 [term 0 FOLLOWER]: Advancing to term 1
I20260809 15:35:57.588012 8442 raft_consensus.cc:504] T 00000000000000000000000000000000 P da58c50a461a416a83f1a0e86f316b77 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "da58c50a461a416a83f1a0e86f316b77" member_type: VOTER last_known_addr { host: "127.3.89.254" port: 38991 } }
I20260809 15:35:57.588343 8442 leader_election.cc:302] T 00000000000000000000000000000000 P da58c50a461a416a83f1a0e86f316b77 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: da58c50a461a416a83f1a0e86f316b77; no voters:
I20260809 15:35:57.588649 8442 leader_election.cc:288] T 00000000000000000000000000000000 P da58c50a461a416a83f1a0e86f316b77 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 15:35:57.588768 8447 raft_consensus.cc:2781] T 00000000000000000000000000000000 P da58c50a461a416a83f1a0e86f316b77 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 15:35:57.589044 8447 raft_consensus.cc:686] T 00000000000000000000000000000000 P da58c50a461a416a83f1a0e86f316b77 [term 1 LEADER]: Becoming Leader. State: Replica: da58c50a461a416a83f1a0e86f316b77, State: Running, Role: LEADER
I20260809 15:35:57.589416 8447 consensus_queue.cc:237] T 00000000000000000000000000000000 P da58c50a461a416a83f1a0e86f316b77 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "da58c50a461a416a83f1a0e86f316b77" member_type: VOTER last_known_addr { host: "127.3.89.254" port: 38991 } }
I20260809 15:35:57.589913 8442 sys_catalog.cc:564] T 00000000000000000000000000000000 P da58c50a461a416a83f1a0e86f316b77 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 15:35:57.591068 8448 sys_catalog.cc:455] T 00000000000000000000000000000000 P da58c50a461a416a83f1a0e86f316b77 [sys.catalog]: SysCatalogTable state changed. Reason: New leader da58c50a461a416a83f1a0e86f316b77. Latest consensus state: current_term: 1 leader_uuid: "da58c50a461a416a83f1a0e86f316b77" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "da58c50a461a416a83f1a0e86f316b77" member_type: VOTER last_known_addr { host: "127.3.89.254" port: 38991 } } }
I20260809 15:35:57.591205 8448 sys_catalog.cc:458] T 00000000000000000000000000000000 P da58c50a461a416a83f1a0e86f316b77 [sys.catalog]: This master's current role is: LEADER
I20260809 15:35:57.591586 8457 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 15:35:57.591603 8449 sys_catalog.cc:455] T 00000000000000000000000000000000 P da58c50a461a416a83f1a0e86f316b77 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "da58c50a461a416a83f1a0e86f316b77" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "da58c50a461a416a83f1a0e86f316b77" member_type: VOTER last_known_addr { host: "127.3.89.254" port: 38991 } } }
I20260809 15:35:57.591831 8449 sys_catalog.cc:458] T 00000000000000000000000000000000 P da58c50a461a416a83f1a0e86f316b77 [sys.catalog]: This master's current role is: LEADER
I20260809 15:35:57.594461 8457 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 15:35:57.600319 8457 catalog_manager.cc:1347] Generated new cluster ID: b91f1857c05540968d719ad1198db8b4
I20260809 15:35:57.600395 8457 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 15:35:57.610448 8457 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 15:35:57.610774 8457 catalog_manager.cc:1504] Loading token signing keys...
I20260809 15:35:57.618553 8457 catalog_manager.cc:5868] T 00000000000000000000000000000000 P da58c50a461a416a83f1a0e86f316b77: Generated new TSK 0
I20260809 15:35:57.619403 8457 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260809 15:35:57.627617 3431 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskm314YR/build/debug/bin/kudu
/tmp/dist-test-taskm314YR/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786289732823074-3431-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786289732823074-3431-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786289732823074-3431-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786289732823074-3431-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.3.89.193:0
--local_ip_for_outbound_sockets=127.3.89.193
--webserver_interface=127.3.89.193
--webserver_port=0
--tserver_master_addrs=127.3.89.254:38991
--builtin_ntp_servers=127.3.89.212:44115
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled
--rpc_encryption=disabled with env {}
W20260809 15:35:57.741677 8468 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 15:35:57.741920 8468 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 15:35:57.741961 8468 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 15:35:57.746025 8468 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 15:35:57.746079 8468 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 15:35:57.746133 8468 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.3.89.193
I20260809 15:35:57.750677 8468 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.3.89.212:44115
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786289732823074-3431-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786289732823074-3431-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.3.89.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786289732823074-3431-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.3.89.193
--webserver_port=0
--tserver_master_addrs=127.3.89.254:38991
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.8468
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.3.89.193
--log_dir=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786289732823074-3431-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 09 Aug 2026 14:46:55 UTC on ccycloud.quasar-yswkmf.root.comops.site
build id 4576540
I20260809 15:35:57.751716 8468 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 15:35:57.753000 8468 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260809 15:35:57.762428 8468 server_base.cc:1034] running on GCE node
W20260809 15:35:57.762428 8476 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 15:35:57.762398 8474 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 15:35:57.762401 8473 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 15:35:57.763136 8468 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 15:35:57.763761 8468 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 15:35:57.764945 8468 hybrid_clock.cc:648] HybridClock initialized: now 1786289757764912 us; error 50 us; skew 500 ppm
I20260809 15:35:57.766748 8468 webserver.cc:460] Webserver started at http://127.3.89.193:41943/ using document root <none> and password file <none>
I20260809 15:35:57.767385 8468 fs_manager.cc:359] Metadata directory not provided
I20260809 15:35:57.767472 8468 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 15:35:57.767668 8468 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 15:35:57.769585 8468 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786289732823074-3431-0/minicluster-data/ts-0/data/instance:
uuid: "8ea3307ea55f4c2f96b12f5883d28751"
format_stamp: "Formatted at 2026-08-09 15:35:57 on dist-test-slave-2w3w"
server_key: "1fdf83a604b7c1ca433239f549ac5e20"
server_key_iv: "510e3ce20907e4bb644171982cf5ebe4"
server_key_version: "encryptionkey@0"
I20260809 15:35:57.770138 8468 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786289732823074-3431-0/minicluster-data/ts-0/wal/instance:
uuid: "8ea3307ea55f4c2f96b12f5883d28751"
format_stamp: "Formatted at 2026-08-09 15:35:57 on dist-test-slave-2w3w"
server_key: "1fdf83a604b7c1ca433239f549ac5e20"
server_key_iv: "510e3ce20907e4bb644171982cf5ebe4"
server_key_version: "encryptionkey@0"
I20260809 15:35:57.773686 8468 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.003s sys 0.003s
I20260809 15:35:57.776072 8484 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 15:35:57.777237 8468 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260809 15:35:57.777385 8468 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786289732823074-3431-0/minicluster-data/ts-0/data,/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786289732823074-3431-0/minicluster-data/ts-0/wal
uuid: "8ea3307ea55f4c2f96b12f5883d28751"
format_stamp: "Formatted at 2026-08-09 15:35:57 on dist-test-slave-2w3w"
server_key: "1fdf83a604b7c1ca433239f549ac5e20"
server_key_iv: "510e3ce20907e4bb644171982cf5ebe4"
server_key_version: "encryptionkey@0"
I20260809 15:35:57.777493 8468 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786289732823074-3431-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786289732823074-3431-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786289732823074-3431-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 15:35:57.781850 8468 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 15:35:57.782409 8468 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 15:35:57.782605 8468 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 15:35:57.783190 8468 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 15:35:57.784058 8468 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 15:35:57.784127 8468 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 15:35:57.784195 8468 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 15:35:57.784237 8468 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 15:35:57.793239 8468 rpc_server.cc:307] RPC server started. Bound to: 127.3.89.193:33727
I20260809 15:35:57.793263 8601 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.3.89.193:33727 every 8 connection(s)
I20260809 15:35:57.794104 8468 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786289732823074-3431-0/minicluster-data/ts-0/data/info.pb
I20260809 15:35:57.794663 3431 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskm314YR/build/debug/bin/kudu as pid 8468
I20260809 15:35:57.794849 3431 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786289732823074-3431-0/minicluster-data/ts-0/wal/instance
I20260809 15:35:57.795192 3431 external_mini_cluster.cc:1428] Setting key 35f5a98c2e9debe0691813df6386740a
I20260809 15:35:57.797287 3431 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskm314YR/build/debug/bin/kudu
/tmp/dist-test-taskm314YR/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786289732823074-3431-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786289732823074-3431-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786289732823074-3431-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786289732823074-3431-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.3.89.194:0
--local_ip_for_outbound_sockets=127.3.89.194
--webserver_interface=127.3.89.194
--webserver_port=0
--tserver_master_addrs=127.3.89.254:38991
--builtin_ntp_servers=127.3.89.212:44115
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled
--rpc_encryption=disabled with env {}
I20260809 15:35:57.801353 8445 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:35:57.796073 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.89.193:42057 (local address 127.3.89.254:38991)
0809 15:35:57.796216 (+ 143us) server_negotiation.cc:206] Beginning negotiation
0809 15:35:57.796221 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 15:35:57.797467 (+ 1246us) server_negotiation.cc:407] Connection header received
0809 15:35:57.799550 (+ 2083us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 15:35:57.799556 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 15:35:57.799602 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 15:35:57.799707 (+ 105us) server_negotiation.cc:226] Negotiated authn=SASL
0809 15:35:57.800898 (+ 1191us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 15:35:57.800901 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 15:35:57.800903 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 15:35:57.800915 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0809 15:35:57.800935 (+ 20us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 15:35:57.800942 (+ 7us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 15:35:57.801023 (+ 81us) server_negotiation.cc:1035] Waiting for connection context
0809 15:35:57.801195 (+ 172us) server_negotiation.cc:299] Negotiation successful
0809 15:35:57.801234 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":60}
I20260809 15:35:57.802480 8605 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:35:57.796517 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.89.254:38991 (local address 127.3.89.193:42057)
0809 15:35:57.797248 (+ 731us) negotiation.cc:107] Waiting for socket to connect
0809 15:35:57.797334 (+ 86us) client_negotiation.cc:174] Beginning negotiation
0809 15:35:57.799303 (+ 1969us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 15:35:57.799858 (+ 555us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 15:35:57.799863 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 15:35:57.799944 (+ 81us) client_negotiation.cc:189] Negotiated authn=SASL
0809 15:35:57.800576 (+ 632us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0809 15:35:57.800581 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0809 15:35:57.800634 (+ 53us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0809 15:35:57.800720 (+ 86us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 15:35:57.801036 (+ 316us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 15:35:57.801048 (+ 12us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 15:35:57.801057 (+ 9us) client_negotiation.cc:769] Sending connection context
0809 15:35:57.801229 (+ 172us) client_negotiation.cc:240] Negotiation successful
0809 15:35:57.801467 (+ 238us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":267,"thread_start_us":129,"threads_started":1}
I20260809 15:35:57.803833 8602 heartbeater.cc:347] Connected to a master server at 127.3.89.254:38991
I20260809 15:35:57.804059 8602 heartbeater.cc:464] Registering TS with master...
I20260809 15:35:57.804471 8602 heartbeater.cc:511] Master 127.3.89.254:38991 requested a full tablet report, sending...
I20260809 15:35:57.806144 8406 ts_manager.cc:194] Registered new tserver with Master: 8ea3307ea55f4c2f96b12f5883d28751 (127.3.89.193:33727)
W20260809 15:35:57.909154 8606 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 15:35:57.909418 8606 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 15:35:57.909458 8606 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 15:35:57.913319 8606 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 15:35:57.913360 8606 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 15:35:57.913412 8606 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.3.89.194
I20260809 15:35:57.917932 8606 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.3.89.212:44115
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786289732823074-3431-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786289732823074-3431-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.3.89.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786289732823074-3431-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.3.89.194
--webserver_port=0
--tserver_master_addrs=127.3.89.254:38991
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.8606
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.3.89.194
--log_dir=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786289732823074-3431-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 09 Aug 2026 14:46:55 UTC on ccycloud.quasar-yswkmf.root.comops.site
build id 4576540
I20260809 15:35:57.918890 8606 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 15:35:57.920356 8606 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 15:35:57.929885 8614 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 15:35:57.929901 8611 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 15:35:57.929870 8612 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 15:35:57.930042 8606 server_base.cc:1034] running on GCE node
I20260809 15:35:57.930538 8606 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 15:35:57.931227 8606 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 15:35:57.932444 8606 hybrid_clock.cc:648] HybridClock initialized: now 1786289757932432 us; error 25 us; skew 500 ppm
I20260809 15:35:57.934376 8606 webserver.cc:460] Webserver started at http://127.3.89.194:38019/ using document root <none> and password file <none>
I20260809 15:35:57.935067 8606 fs_manager.cc:359] Metadata directory not provided
I20260809 15:35:57.935163 8606 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 15:35:57.935403 8606 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 15:35:57.937438 8606 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786289732823074-3431-0/minicluster-data/ts-1/data/instance:
uuid: "39de4f0f5422417e886051530c80aa17"
format_stamp: "Formatted at 2026-08-09 15:35:57 on dist-test-slave-2w3w"
server_key: "76920d406d5d5989f72ca43d38c98bc6"
server_key_iv: "d5d21f166fa019fa216d5cb048a2e250"
server_key_version: "encryptionkey@0"
I20260809 15:35:57.938009 8606 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786289732823074-3431-0/minicluster-data/ts-1/wal/instance:
uuid: "39de4f0f5422417e886051530c80aa17"
format_stamp: "Formatted at 2026-08-09 15:35:57 on dist-test-slave-2w3w"
server_key: "76920d406d5d5989f72ca43d38c98bc6"
server_key_iv: "d5d21f166fa019fa216d5cb048a2e250"
server_key_version: "encryptionkey@0"
I20260809 15:35:57.941603 8606 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.002s sys 0.004s
I20260809 15:35:57.944120 8621 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 15:35:57.945367 8606 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260809 15:35:57.945500 8606 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786289732823074-3431-0/minicluster-data/ts-1/data,/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786289732823074-3431-0/minicluster-data/ts-1/wal
uuid: "39de4f0f5422417e886051530c80aa17"
format_stamp: "Formatted at 2026-08-09 15:35:57 on dist-test-slave-2w3w"
server_key: "76920d406d5d5989f72ca43d38c98bc6"
server_key_iv: "d5d21f166fa019fa216d5cb048a2e250"
server_key_version: "encryptionkey@0"
I20260809 15:35:57.945613 8606 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786289732823074-3431-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786289732823074-3431-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786289732823074-3431-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 15:35:57.950068 8606 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 15:35:57.950631 8606 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 15:35:57.950824 8606 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 15:35:57.951418 8606 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 15:35:57.952284 8606 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 15:35:57.952354 8606 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 15:35:57.952420 8606 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 15:35:57.952469 8606 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 15:35:57.961230 8606 rpc_server.cc:307] RPC server started. Bound to: 127.3.89.194:44053
I20260809 15:35:57.961256 8741 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.3.89.194:44053 every 8 connection(s)
I20260809 15:35:57.962093 8606 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786289732823074-3431-0/minicluster-data/ts-1/data/info.pb
I20260809 15:35:57.965157 3431 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskm314YR/build/debug/bin/kudu as pid 8606
I20260809 15:35:57.965312 3431 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786289732823074-3431-0/minicluster-data/ts-1/wal/instance
I20260809 15:35:57.965598 3431 external_mini_cluster.cc:1428] Setting key 5cb8276a477773a3dd068e1712e3a1ec
I20260809 15:35:57.967784 3431 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskm314YR/build/debug/bin/kudu
/tmp/dist-test-taskm314YR/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786289732823074-3431-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786289732823074-3431-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786289732823074-3431-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786289732823074-3431-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.3.89.195:0
--local_ip_for_outbound_sockets=127.3.89.195
--webserver_interface=127.3.89.195
--webserver_port=0
--tserver_master_addrs=127.3.89.254:38991
--builtin_ntp_servers=127.3.89.212:44115
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled
--rpc_encryption=disabled with env {}
I20260809 15:35:57.968652 8445 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:35:57.964015 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.89.194:54057 (local address 127.3.89.254:38991)
0809 15:35:57.964168 (+ 153us) server_negotiation.cc:206] Beginning negotiation
0809 15:35:57.964172 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 15:35:57.965260 (+ 1088us) server_negotiation.cc:407] Connection header received
0809 15:35:57.966751 (+ 1491us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 15:35:57.966755 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 15:35:57.966795 (+ 40us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 15:35:57.966866 (+ 71us) server_negotiation.cc:226] Negotiated authn=SASL
0809 15:35:57.968127 (+ 1261us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 15:35:57.968134 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 15:35:57.968139 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 15:35:57.968156 (+ 17us) server_negotiation.cc:922] calling sasl_server_start()
0809 15:35:57.968184 (+ 28us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 15:35:57.968197 (+ 13us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 15:35:57.968308 (+ 111us) server_negotiation.cc:1035] Waiting for connection context
0809 15:35:57.968416 (+ 108us) server_negotiation.cc:299] Negotiation successful
0809 15:35:57.968465 (+ 49us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":64}
I20260809 15:35:57.969409 8744 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:35:57.964376 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.89.254:38991 (local address 127.3.89.194:54057)
0809 15:35:57.965088 (+ 712us) negotiation.cc:107] Waiting for socket to connect
0809 15:35:57.965129 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0809 15:35:57.966545 (+ 1416us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 15:35:57.967079 (+ 534us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 15:35:57.967085 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 15:35:57.967171 (+ 86us) client_negotiation.cc:189] Negotiated authn=SASL
0809 15:35:57.967816 (+ 645us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0809 15:35:57.967817 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0809 15:35:57.967856 (+ 39us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0809 15:35:57.967951 (+ 95us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 15:35:57.968289 (+ 338us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 15:35:57.968296 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 15:35:57.968299 (+ 3us) client_negotiation.cc:769] Sending connection context
0809 15:35:57.968428 (+ 129us) client_negotiation.cc:240] Negotiation successful
0809 15:35:57.968608 (+ 180us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":277,"thread_start_us":117,"threads_started":1}
I20260809 15:35:57.970542 8742 heartbeater.cc:347] Connected to a master server at 127.3.89.254:38991
I20260809 15:35:57.970778 8742 heartbeater.cc:464] Registering TS with master...
I20260809 15:35:57.971230 8742 heartbeater.cc:511] Master 127.3.89.254:38991 requested a full tablet report, sending...
I20260809 15:35:57.972234 8406 ts_manager.cc:194] Registered new tserver with Master: 39de4f0f5422417e886051530c80aa17 (127.3.89.194:44053)
W20260809 15:35:58.079733 8745 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 15:35:58.080013 8745 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 15:35:58.080084 8745 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 15:35:58.084295 8745 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 15:35:58.084373 8745 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 15:35:58.084479 8745 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.3.89.195
I20260809 15:35:58.089284 8745 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.3.89.212:44115
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786289732823074-3431-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786289732823074-3431-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.3.89.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786289732823074-3431-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.3.89.195
--webserver_port=0
--tserver_master_addrs=127.3.89.254:38991
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.8745
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.3.89.195
--log_dir=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786289732823074-3431-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 09 Aug 2026 14:46:55 UTC on ccycloud.quasar-yswkmf.root.comops.site
build id 4576540
I20260809 15:35:58.090374 8745 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 15:35:58.091878 8745 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 15:35:58.101490 8752 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 15:35:58.101529 8751 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 15:35:58.101492 8754 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 15:35:58.101538 8745 server_base.cc:1034] running on GCE node
I20260809 15:35:58.102058 8745 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 15:35:58.102706 8745 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 15:35:58.103866 8745 hybrid_clock.cc:648] HybridClock initialized: now 1786289758103848 us; error 30 us; skew 500 ppm
I20260809 15:35:58.105701 8745 webserver.cc:460] Webserver started at http://127.3.89.195:38881/ using document root <none> and password file <none>
I20260809 15:35:58.106295 8745 fs_manager.cc:359] Metadata directory not provided
I20260809 15:35:58.106357 8745 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 15:35:58.106621 8745 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 15:35:58.108688 8745 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786289732823074-3431-0/minicluster-data/ts-2/data/instance:
uuid: "cfe93bb4592d455c87f61dfd5ca87ddb"
format_stamp: "Formatted at 2026-08-09 15:35:58 on dist-test-slave-2w3w"
server_key: "4b96a785e8b18e1ca8e100f4ea4b1516"
server_key_iv: "4c6b66235a8b15657da985a2142b8e7d"
server_key_version: "encryptionkey@0"
I20260809 15:35:58.109241 8745 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786289732823074-3431-0/minicluster-data/ts-2/wal/instance:
uuid: "cfe93bb4592d455c87f61dfd5ca87ddb"
format_stamp: "Formatted at 2026-08-09 15:35:58 on dist-test-slave-2w3w"
server_key: "4b96a785e8b18e1ca8e100f4ea4b1516"
server_key_iv: "4c6b66235a8b15657da985a2142b8e7d"
server_key_version: "encryptionkey@0"
I20260809 15:35:58.112857 8745 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.005s sys 0.000s
I20260809 15:35:58.115175 8762 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 15:35:58.116369 8745 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260809 15:35:58.116492 8745 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786289732823074-3431-0/minicluster-data/ts-2/data,/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786289732823074-3431-0/minicluster-data/ts-2/wal
uuid: "cfe93bb4592d455c87f61dfd5ca87ddb"
format_stamp: "Formatted at 2026-08-09 15:35:58 on dist-test-slave-2w3w"
server_key: "4b96a785e8b18e1ca8e100f4ea4b1516"
server_key_iv: "4c6b66235a8b15657da985a2142b8e7d"
server_key_version: "encryptionkey@0"
I20260809 15:35:58.116600 8745 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786289732823074-3431-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786289732823074-3431-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786289732823074-3431-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 15:35:58.121227 8745 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 15:35:58.121816 8745 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 15:35:58.121999 8745 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 15:35:58.122536 8745 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 15:35:58.123519 8745 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 15:35:58.123566 8745 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 15:35:58.123632 8745 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 15:35:58.123673 8745 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 15:35:58.131948 8745 rpc_server.cc:307] RPC server started. Bound to: 127.3.89.195:41025
I20260809 15:35:58.131970 8883 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.3.89.195:41025 every 8 connection(s)
I20260809 15:35:58.132786 8745 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786289732823074-3431-0/minicluster-data/ts-2/data/info.pb
I20260809 15:35:58.135208 3431 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskm314YR/build/debug/bin/kudu as pid 8745
I20260809 15:35:58.135322 3431 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786289732823074-3431-0/minicluster-data/ts-2/wal/instance
I20260809 15:35:58.135637 3431 external_mini_cluster.cc:1428] Setting key 61bc8dafc29ba43682cb2adec0613f3c
I20260809 15:35:58.139199 8445 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:35:58.134550 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.89.195:55667 (local address 127.3.89.254:38991)
0809 15:35:58.134690 (+ 140us) server_negotiation.cc:206] Beginning negotiation
0809 15:35:58.134694 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 15:35:58.136056 (+ 1362us) server_negotiation.cc:407] Connection header received
0809 15:35:58.137454 (+ 1398us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 15:35:58.137458 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 15:35:58.137504 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 15:35:58.137608 (+ 104us) server_negotiation.cc:226] Negotiated authn=SASL
0809 15:35:58.138700 (+ 1092us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 15:35:58.138704 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 15:35:58.138706 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 15:35:58.138718 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0809 15:35:58.138740 (+ 22us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 15:35:58.138752 (+ 12us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 15:35:58.138857 (+ 105us) server_negotiation.cc:1035] Waiting for connection context
0809 15:35:58.139027 (+ 170us) server_negotiation.cc:299] Negotiation successful
0809 15:35:58.139074 (+ 47us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":51}
I20260809 15:35:58.139986 8886 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:35:58.135022 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.89.254:38991 (local address 127.3.89.195:55667)
0809 15:35:58.135883 (+ 861us) negotiation.cc:107] Waiting for socket to connect
0809 15:35:58.135936 (+ 53us) client_negotiation.cc:174] Beginning negotiation
0809 15:35:58.137233 (+ 1297us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 15:35:58.137769 (+ 536us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 15:35:58.137775 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 15:35:58.137866 (+ 91us) client_negotiation.cc:189] Negotiated authn=SASL
0809 15:35:58.138455 (+ 589us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0809 15:35:58.138456 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0809 15:35:58.138493 (+ 37us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0809 15:35:58.138556 (+ 63us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 15:35:58.138847 (+ 291us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 15:35:58.138855 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 15:35:58.138861 (+ 6us) client_negotiation.cc:769] Sending connection context
0809 15:35:58.139062 (+ 201us) client_negotiation.cc:240] Negotiation successful
0809 15:35:58.139300 (+ 238us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":377,"thread_start_us":161,"threads_started":1}
I20260809 15:35:58.140969 8884 heartbeater.cc:347] Connected to a master server at 127.3.89.254:38991
I20260809 15:35:58.141185 8884 heartbeater.cc:464] Registering TS with master...
I20260809 15:35:58.141566 8884 heartbeater.cc:511] Master 127.3.89.254:38991 requested a full tablet report, sending...
I20260809 15:35:58.142490 8406 ts_manager.cc:194] Registered new tserver with Master: cfe93bb4592d455c87f61dfd5ca87ddb (127.3.89.195:41025)
I20260809 15:35:58.149292 3431 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260809 15:35:58.153177 8445 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:35:58.151333 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51112 (local address 127.3.89.254:38991)
0809 15:35:58.151459 (+ 126us) server_negotiation.cc:206] Beginning negotiation
0809 15:35:58.151463 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 15:35:58.151775 (+ 312us) server_negotiation.cc:407] Connection header received
0809 15:35:58.151903 (+ 128us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 15:35:58.151905 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 15:35:58.151937 (+ 32us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 15:35:58.152016 (+ 79us) server_negotiation.cc:226] Negotiated authn=SASL
0809 15:35:58.152752 (+ 736us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 15:35:58.152755 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 15:35:58.152756 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 15:35:58.152767 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0809 15:35:58.152785 (+ 18us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 15:35:58.152792 (+ 7us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 15:35:58.152862 (+ 70us) server_negotiation.cc:1035] Waiting for connection context
0809 15:35:58.152985 (+ 123us) server_negotiation.cc:299] Negotiation successful
0809 15:35:58.153020 (+ 35us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
I20260809 15:35:58.156121 8406 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:51112:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260809 15:35:58.159477 8406 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260809 15:35:58.172030 8900 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:35:58.168482 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.89.194:44053 (local address 127.0.0.1:55014)
0809 15:35:58.169280 (+ 798us) negotiation.cc:107] Waiting for socket to connect
0809 15:35:58.169329 (+ 49us) client_negotiation.cc:174] Beginning negotiation
0809 15:35:58.169447 (+ 118us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 15:35:58.170427 (+ 980us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 15:35:58.170433 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 15:35:58.170450 (+ 17us) client_negotiation.cc:189] Negotiated authn=SASL
0809 15:35:58.171085 (+ 635us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0809 15:35:58.171087 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0809 15:35:58.171099 (+ 12us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0809 15:35:58.171126 (+ 27us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 15:35:58.171634 (+ 508us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 15:35:58.171646 (+ 12us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 15:35:58.171653 (+ 7us) client_negotiation.cc:769] Sending connection context
0809 15:35:58.171777 (+ 124us) client_negotiation.cc:240] Negotiation successful
0809 15:35:58.171851 (+ 74us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":679,"spinlock_wait_cycles":13440,"thread_start_us":115,"threads_started":1}
I20260809 15:35:58.172073 8896 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:35:58.168405 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.89.195:41025 (local address 127.0.0.1:36544)
0809 15:35:58.169277 (+ 872us) negotiation.cc:107] Waiting for socket to connect
0809 15:35:58.169328 (+ 51us) client_negotiation.cc:174] Beginning negotiation
0809 15:35:58.169447 (+ 119us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 15:35:58.170228 (+ 781us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 15:35:58.170233 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 15:35:58.170307 (+ 74us) client_negotiation.cc:189] Negotiated authn=SASL
0809 15:35:58.170917 (+ 610us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0809 15:35:58.170921 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0809 15:35:58.170944 (+ 23us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0809 15:35:58.171059 (+ 115us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 15:35:58.171887 (+ 828us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 15:35:58.171890 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 15:35:58.171893 (+ 3us) client_negotiation.cc:769] Sending connection context
0809 15:35:58.171944 (+ 51us) client_negotiation.cc:240] Negotiation successful
0809 15:35:58.171972 (+ 28us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":696,"spinlock_wait_cycles":4352,"thread_start_us":83,"threads_started":1}
I20260809 15:35:58.172376 8901 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:35:58.169193 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33848 (local address 127.3.89.193:33727)
0809 15:35:58.169978 (+ 785us) server_negotiation.cc:206] Beginning negotiation
0809 15:35:58.169989 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0809 15:35:58.170010 (+ 21us) server_negotiation.cc:407] Connection header received
0809 15:35:58.170064 (+ 54us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 15:35:58.170069 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 15:35:58.170214 (+ 145us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 15:35:58.170327 (+ 113us) server_negotiation.cc:226] Negotiated authn=SASL
0809 15:35:58.171614 (+ 1287us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 15:35:58.171620 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 15:35:58.171629 (+ 9us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 15:35:58.171641 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0809 15:35:58.171684 (+ 43us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 15:35:58.171722 (+ 38us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 15:35:58.171850 (+ 128us) server_negotiation.cc:1035] Waiting for connection context
0809 15:35:58.172101 (+ 251us) server_negotiation.cc:299] Negotiation successful
0809 15:35:58.172172 (+ 71us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":678,"thread_start_us":129,"threads_started":1}
I20260809 15:35:58.172739 8897 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:35:58.168765 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.89.193:33727 (local address 127.0.0.1:33848)
0809 15:35:58.169645 (+ 880us) negotiation.cc:107] Waiting for socket to connect
0809 15:35:58.169658 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0809 15:35:58.169724 (+ 66us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 15:35:58.170943 (+ 1219us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 15:35:58.170947 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 15:35:58.170994 (+ 47us) client_negotiation.cc:189] Negotiated authn=SASL
0809 15:35:58.171425 (+ 431us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0809 15:35:58.171427 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0809 15:35:58.171438 (+ 11us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0809 15:35:58.171466 (+ 28us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 15:35:58.171897 (+ 431us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 15:35:58.171900 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 15:35:58.171904 (+ 4us) client_negotiation.cc:769] Sending connection context
0809 15:35:58.171978 (+ 74us) client_negotiation.cc:240] Negotiation successful
0809 15:35:58.172012 (+ 34us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":819,"thread_start_us":97,"threads_started":1}
I20260809 15:35:58.172752 8902 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:35:58.169268 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55014 (local address 127.3.89.194:44053)
0809 15:35:58.169990 (+ 722us) server_negotiation.cc:206] Beginning negotiation
0809 15:35:58.170010 (+ 20us) server_negotiation.cc:399] Waiting for connection header
0809 15:35:58.170030 (+ 20us) server_negotiation.cc:407] Connection header received
0809 15:35:58.170082 (+ 52us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 15:35:58.170086 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 15:35:58.170268 (+ 182us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 15:35:58.170655 (+ 387us) server_negotiation.cc:226] Negotiated authn=SASL
0809 15:35:58.171430 (+ 775us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 15:35:58.171434 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 15:35:58.171442 (+ 8us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 15:35:58.171449 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0809 15:35:58.171479 (+ 30us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 15:35:58.171494 (+ 15us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 15:35:58.171658 (+ 164us) server_negotiation.cc:1035] Waiting for connection context
0809 15:35:58.171802 (+ 144us) server_negotiation.cc:299] Negotiation successful
0809 15:35:58.171890 (+ 88us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":571,"thread_start_us":145,"threads_started":1}
I20260809 15:35:58.173055 8898 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:35:58.168517 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36544 (local address 127.3.89.195:41025)
0809 15:35:58.169759 (+ 1242us) server_negotiation.cc:206] Beginning negotiation
0809 15:35:58.169773 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0809 15:35:58.169798 (+ 25us) server_negotiation.cc:407] Connection header received
0809 15:35:58.169858 (+ 60us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 15:35:58.169863 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 15:35:58.170065 (+ 202us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 15:35:58.170204 (+ 139us) server_negotiation.cc:226] Negotiated authn=SASL
0809 15:35:58.171674 (+ 1470us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 15:35:58.171680 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 15:35:58.171690 (+ 10us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 15:35:58.171703 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0809 15:35:58.171752 (+ 49us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 15:35:58.171783 (+ 31us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 15:35:58.172184 (+ 401us) server_negotiation.cc:1035] Waiting for connection context
0809 15:35:58.172229 (+ 45us) server_negotiation.cc:299] Negotiation successful
0809 15:35:58.172299 (+ 70us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1132,"thread_start_us":149,"threads_started":1}
I20260809 15:35:58.175732 8673 tablet_service.cc:1467] Processing CreateTablet for tablet 916879440a494083be11fd8782090277 (DEFAULT_TABLE table=test-table [id=2e4a9a33ee4f457b9fd432152e3a190d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 15:35:58.175740 8535 tablet_service.cc:1467] Processing CreateTablet for tablet 916879440a494083be11fd8782090277 (DEFAULT_TABLE table=test-table [id=2e4a9a33ee4f457b9fd432152e3a190d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 15:35:58.175740 8814 tablet_service.cc:1467] Processing CreateTablet for tablet 916879440a494083be11fd8782090277 (DEFAULT_TABLE table=test-table [id=2e4a9a33ee4f457b9fd432152e3a190d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 15:35:58.176748 8673 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 916879440a494083be11fd8782090277. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 15:35:58.176771 8535 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 916879440a494083be11fd8782090277. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 15:35:58.176771 8814 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 916879440a494083be11fd8782090277. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 15:35:58.184058 8904 tablet_bootstrap.cc:492] T 916879440a494083be11fd8782090277 P 8ea3307ea55f4c2f96b12f5883d28751: Bootstrap starting.
I20260809 15:35:58.184492 8903 tablet_bootstrap.cc:492] T 916879440a494083be11fd8782090277 P 39de4f0f5422417e886051530c80aa17: Bootstrap starting.
I20260809 15:35:58.184486 8905 tablet_bootstrap.cc:492] T 916879440a494083be11fd8782090277 P cfe93bb4592d455c87f61dfd5ca87ddb: Bootstrap starting.
I20260809 15:35:58.186398 8903 tablet_bootstrap.cc:654] T 916879440a494083be11fd8782090277 P 39de4f0f5422417e886051530c80aa17: Neither blocks nor log segments found. Creating new log.
I20260809 15:35:58.186400 8905 tablet_bootstrap.cc:654] T 916879440a494083be11fd8782090277 P cfe93bb4592d455c87f61dfd5ca87ddb: Neither blocks nor log segments found. Creating new log.
I20260809 15:35:58.186400 8904 tablet_bootstrap.cc:654] T 916879440a494083be11fd8782090277 P 8ea3307ea55f4c2f96b12f5883d28751: Neither blocks nor log segments found. Creating new log.
I20260809 15:35:58.187323 8903 log.cc:824] T 916879440a494083be11fd8782090277 P 39de4f0f5422417e886051530c80aa17: Log is configured to *not* fsync() on all Append() calls
I20260809 15:35:58.187326 8905 log.cc:824] T 916879440a494083be11fd8782090277 P cfe93bb4592d455c87f61dfd5ca87ddb: Log is configured to *not* fsync() on all Append() calls
I20260809 15:35:58.187326 8904 log.cc:824] T 916879440a494083be11fd8782090277 P 8ea3307ea55f4c2f96b12f5883d28751: Log is configured to *not* fsync() on all Append() calls
I20260809 15:35:58.189507 8903 tablet_bootstrap.cc:492] T 916879440a494083be11fd8782090277 P 39de4f0f5422417e886051530c80aa17: No bootstrap required, opened a new log
I20260809 15:35:58.189540 8904 tablet_bootstrap.cc:492] T 916879440a494083be11fd8782090277 P 8ea3307ea55f4c2f96b12f5883d28751: No bootstrap required, opened a new log
I20260809 15:35:58.189725 8903 ts_tablet_manager.cc:1397] T 916879440a494083be11fd8782090277 P 39de4f0f5422417e886051530c80aa17: Time spent bootstrapping tablet: real 0.005s user 0.003s sys 0.000s
I20260809 15:35:58.189729 8904 ts_tablet_manager.cc:1397] T 916879440a494083be11fd8782090277 P 8ea3307ea55f4c2f96b12f5883d28751: Time spent bootstrapping tablet: real 0.006s user 0.004s sys 0.000s
I20260809 15:35:58.190111 8905 tablet_bootstrap.cc:492] T 916879440a494083be11fd8782090277 P cfe93bb4592d455c87f61dfd5ca87ddb: No bootstrap required, opened a new log
I20260809 15:35:58.190322 8905 ts_tablet_manager.cc:1397] T 916879440a494083be11fd8782090277 P cfe93bb4592d455c87f61dfd5ca87ddb: Time spent bootstrapping tablet: real 0.006s user 0.005s sys 0.000s
I20260809 15:35:58.194514 8903 raft_consensus.cc:348] T 916879440a494083be11fd8782090277 P 39de4f0f5422417e886051530c80aa17 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cfe93bb4592d455c87f61dfd5ca87ddb" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41025 } } peers { permanent_uuid: "39de4f0f5422417e886051530c80aa17" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 44053 } } peers { permanent_uuid: "8ea3307ea55f4c2f96b12f5883d28751" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 33727 } }
I20260809 15:35:58.194768 8903 raft_consensus.cc:374] T 916879440a494083be11fd8782090277 P 39de4f0f5422417e886051530c80aa17 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 15:35:58.194587 8904 raft_consensus.cc:348] T 916879440a494083be11fd8782090277 P 8ea3307ea55f4c2f96b12f5883d28751 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cfe93bb4592d455c87f61dfd5ca87ddb" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41025 } } peers { permanent_uuid: "39de4f0f5422417e886051530c80aa17" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 44053 } } peers { permanent_uuid: "8ea3307ea55f4c2f96b12f5883d28751" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 33727 } }
I20260809 15:35:58.194587 8905 raft_consensus.cc:348] T 916879440a494083be11fd8782090277 P cfe93bb4592d455c87f61dfd5ca87ddb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cfe93bb4592d455c87f61dfd5ca87ddb" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41025 } } peers { permanent_uuid: "39de4f0f5422417e886051530c80aa17" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 44053 } } peers { permanent_uuid: "8ea3307ea55f4c2f96b12f5883d28751" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 33727 } }
I20260809 15:35:58.194852 8904 raft_consensus.cc:374] T 916879440a494083be11fd8782090277 P 8ea3307ea55f4c2f96b12f5883d28751 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 15:35:58.194852 8905 raft_consensus.cc:374] T 916879440a494083be11fd8782090277 P cfe93bb4592d455c87f61dfd5ca87ddb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 15:35:58.194864 8903 raft_consensus.cc:729] T 916879440a494083be11fd8782090277 P 39de4f0f5422417e886051530c80aa17 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 39de4f0f5422417e886051530c80aa17, State: Initialized, Role: FOLLOWER
I20260809 15:35:58.194916 8905 raft_consensus.cc:729] T 916879440a494083be11fd8782090277 P cfe93bb4592d455c87f61dfd5ca87ddb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: cfe93bb4592d455c87f61dfd5ca87ddb, State: Initialized, Role: FOLLOWER
I20260809 15:35:58.194986 8904 raft_consensus.cc:729] T 916879440a494083be11fd8782090277 P 8ea3307ea55f4c2f96b12f5883d28751 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8ea3307ea55f4c2f96b12f5883d28751, State: Initialized, Role: FOLLOWER
I20260809 15:35:58.195415 8903 consensus_queue.cc:260] T 916879440a494083be11fd8782090277 P 39de4f0f5422417e886051530c80aa17 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cfe93bb4592d455c87f61dfd5ca87ddb" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41025 } } peers { permanent_uuid: "39de4f0f5422417e886051530c80aa17" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 44053 } } peers { permanent_uuid: "8ea3307ea55f4c2f96b12f5883d28751" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 33727 } }
I20260809 15:35:58.195420 8905 consensus_queue.cc:260] T 916879440a494083be11fd8782090277 P cfe93bb4592d455c87f61dfd5ca87ddb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cfe93bb4592d455c87f61dfd5ca87ddb" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41025 } } peers { permanent_uuid: "39de4f0f5422417e886051530c80aa17" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 44053 } } peers { permanent_uuid: "8ea3307ea55f4c2f96b12f5883d28751" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 33727 } }
I20260809 15:35:58.195420 8904 consensus_queue.cc:260] T 916879440a494083be11fd8782090277 P 8ea3307ea55f4c2f96b12f5883d28751 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cfe93bb4592d455c87f61dfd5ca87ddb" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41025 } } peers { permanent_uuid: "39de4f0f5422417e886051530c80aa17" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 44053 } } peers { permanent_uuid: "8ea3307ea55f4c2f96b12f5883d28751" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 33727 } }
I20260809 15:35:58.196110 8884 heartbeater.cc:503] Master 127.3.89.254:38991 was elected leader, sending a full tablet report...
I20260809 15:35:58.196465 8903 ts_tablet_manager.cc:1428] T 916879440a494083be11fd8782090277 P 39de4f0f5422417e886051530c80aa17: Time spent starting tablet: real 0.007s user 0.007s sys 0.000s
I20260809 15:35:58.196471 8905 ts_tablet_manager.cc:1428] T 916879440a494083be11fd8782090277 P cfe93bb4592d455c87f61dfd5ca87ddb: Time spent starting tablet: real 0.006s user 0.005s sys 0.000s
I20260809 15:35:58.196480 8904 ts_tablet_manager.cc:1428] T 916879440a494083be11fd8782090277 P 8ea3307ea55f4c2f96b12f5883d28751: Time spent starting tablet: real 0.007s user 0.007s sys 0.000s
I20260809 15:35:58.196754 8742 heartbeater.cc:503] Master 127.3.89.254:38991 was elected leader, sending a full tablet report...
I20260809 15:35:58.196781 8602 heartbeater.cc:503] Master 127.3.89.254:38991 was elected leader, sending a full tablet report...
W20260809 15:35:58.213080 8743 tablet.cc:2369] T 916879440a494083be11fd8782090277 P 39de4f0f5422417e886051530c80aa17: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260809 15:35:58.280728 8911 raft_consensus.cc:482] T 916879440a494083be11fd8782090277 P 8ea3307ea55f4c2f96b12f5883d28751 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 15:35:58.281018 8911 raft_consensus.cc:504] T 916879440a494083be11fd8782090277 P 8ea3307ea55f4c2f96b12f5883d28751 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cfe93bb4592d455c87f61dfd5ca87ddb" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41025 } } peers { permanent_uuid: "39de4f0f5422417e886051530c80aa17" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 44053 } } peers { permanent_uuid: "8ea3307ea55f4c2f96b12f5883d28751" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 33727 } }
I20260809 15:35:58.282336 8911 leader_election.cc:288] T 916879440a494083be11fd8782090277 P 8ea3307ea55f4c2f96b12f5883d28751 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers cfe93bb4592d455c87f61dfd5ca87ddb (127.3.89.195:41025), 39de4f0f5422417e886051530c80aa17 (127.3.89.194:44053)
I20260809 15:35:58.284267 8898 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:35:58.282678 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.89.193:41951 (local address 127.3.89.195:41025)
0809 15:35:58.282827 (+ 149us) server_negotiation.cc:206] Beginning negotiation
0809 15:35:58.282834 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 15:35:58.282854 (+ 20us) server_negotiation.cc:407] Connection header received
0809 15:35:58.282908 (+ 54us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 15:35:58.282911 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 15:35:58.282951 (+ 40us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 15:35:58.283083 (+ 132us) server_negotiation.cc:226] Negotiated authn=SASL
0809 15:35:58.283872 (+ 789us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 15:35:58.283875 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 15:35:58.283878 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 15:35:58.283890 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0809 15:35:58.283937 (+ 47us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 15:35:58.283950 (+ 13us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 15:35:58.284048 (+ 98us) server_negotiation.cc:1035] Waiting for connection context
0809 15:35:58.284080 (+ 32us) server_negotiation.cc:299] Negotiation successful
0809 15:35:58.284126 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":71}
I20260809 15:35:58.284286 8605 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:35:58.282548 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.89.195:41025 (local address 127.3.89.193:41951)
0809 15:35:58.282700 (+ 152us) negotiation.cc:107] Waiting for socket to connect
0809 15:35:58.282718 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0809 15:35:58.282820 (+ 102us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 15:35:58.283105 (+ 285us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 15:35:58.283109 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 15:35:58.283123 (+ 14us) client_negotiation.cc:189] Negotiated authn=SASL
0809 15:35:58.283683 (+ 560us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0809 15:35:58.283684 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0809 15:35:58.283694 (+ 10us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0809 15:35:58.283719 (+ 25us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 15:35:58.284027 (+ 308us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 15:35:58.284029 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 15:35:58.284031 (+ 2us) client_negotiation.cc:769] Sending connection context
0809 15:35:58.284079 (+ 48us) client_negotiation.cc:240] Negotiation successful
0809 15:35:58.284133 (+ 54us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":54}
I20260809 15:35:58.284412 8902 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:35:58.282867 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.89.193:49409 (local address 127.3.89.194:44053)
0809 15:35:58.283031 (+ 164us) server_negotiation.cc:206] Beginning negotiation
0809 15:35:58.283036 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 15:35:58.283086 (+ 50us) server_negotiation.cc:407] Connection header received
0809 15:35:58.283210 (+ 124us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 15:35:58.283213 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 15:35:58.283255 (+ 42us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 15:35:58.283335 (+ 80us) server_negotiation.cc:226] Negotiated authn=SASL
0809 15:35:58.284004 (+ 669us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 15:35:58.284010 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 15:35:58.284016 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 15:35:58.284031 (+ 15us) server_negotiation.cc:922] calling sasl_server_start()
0809 15:35:58.284053 (+ 22us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 15:35:58.284065 (+ 12us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 15:35:58.284150 (+ 85us) server_negotiation.cc:1035] Waiting for connection context
0809 15:35:58.284238 (+ 88us) server_negotiation.cc:299] Negotiation successful
0809 15:35:58.284274 (+ 36us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":46}
I20260809 15:35:58.284755 8913 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:35:58.282614 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.89.194:44053 (local address 127.3.89.193:49409)
0809 15:35:58.283016 (+ 402us) negotiation.cc:107] Waiting for socket to connect
0809 15:35:58.283032 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0809 15:35:58.283114 (+ 82us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 15:35:58.283374 (+ 260us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 15:35:58.283377 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 15:35:58.283393 (+ 16us) client_negotiation.cc:189] Negotiated authn=SASL
0809 15:35:58.283859 (+ 466us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0809 15:35:58.283860 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0809 15:35:58.283869 (+ 9us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0809 15:35:58.283893 (+ 24us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 15:35:58.284160 (+ 267us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 15:35:58.284163 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 15:35:58.284166 (+ 3us) client_negotiation.cc:769] Sending connection context
0809 15:35:58.284243 (+ 77us) client_negotiation.cc:240] Negotiation successful
0809 15:35:58.284273 (+ 30us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":316,"thread_start_us":92,"threads_started":1}
I20260809 15:35:58.285079 8834 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "916879440a494083be11fd8782090277" candidate_uuid: "8ea3307ea55f4c2f96b12f5883d28751" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cfe93bb4592d455c87f61dfd5ca87ddb" is_pre_election: true
I20260809 15:35:58.285382 8694 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "916879440a494083be11fd8782090277" candidate_uuid: "8ea3307ea55f4c2f96b12f5883d28751" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "39de4f0f5422417e886051530c80aa17" is_pre_election: true
I20260809 15:35:58.285429 8834 raft_consensus.cc:2454] T 916879440a494083be11fd8782090277 P cfe93bb4592d455c87f61dfd5ca87ddb [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8ea3307ea55f4c2f96b12f5883d28751 in term 0.
I20260809 15:35:58.285652 8694 raft_consensus.cc:2454] T 916879440a494083be11fd8782090277 P 39de4f0f5422417e886051530c80aa17 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8ea3307ea55f4c2f96b12f5883d28751 in term 0.
I20260809 15:35:58.285988 8486 leader_election.cc:302] T 916879440a494083be11fd8782090277 P 8ea3307ea55f4c2f96b12f5883d28751 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8ea3307ea55f4c2f96b12f5883d28751, cfe93bb4592d455c87f61dfd5ca87ddb; no voters:
I20260809 15:35:58.286310 8911 raft_consensus.cc:2781] T 916879440a494083be11fd8782090277 P 8ea3307ea55f4c2f96b12f5883d28751 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260809 15:35:58.286391 8911 raft_consensus.cc:482] T 916879440a494083be11fd8782090277 P 8ea3307ea55f4c2f96b12f5883d28751 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 15:35:58.286422 8911 raft_consensus.cc:3037] T 916879440a494083be11fd8782090277 P 8ea3307ea55f4c2f96b12f5883d28751 [term 0 FOLLOWER]: Advancing to term 1
W20260809 15:35:58.295387 8603 tablet.cc:2369] T 916879440a494083be11fd8782090277 P 8ea3307ea55f4c2f96b12f5883d28751: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260809 15:35:58.342505 8911 raft_consensus.cc:504] T 916879440a494083be11fd8782090277 P 8ea3307ea55f4c2f96b12f5883d28751 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cfe93bb4592d455c87f61dfd5ca87ddb" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41025 } } peers { permanent_uuid: "39de4f0f5422417e886051530c80aa17" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 44053 } } peers { permanent_uuid: "8ea3307ea55f4c2f96b12f5883d28751" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 33727 } }
I20260809 15:35:58.343108 8911 leader_election.cc:288] T 916879440a494083be11fd8782090277 P 8ea3307ea55f4c2f96b12f5883d28751 [CANDIDATE]: Term 1 election: Requested vote from peers cfe93bb4592d455c87f61dfd5ca87ddb (127.3.89.195:41025), 39de4f0f5422417e886051530c80aa17 (127.3.89.194:44053)
I20260809 15:35:58.343626 8694 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "916879440a494083be11fd8782090277" candidate_uuid: "8ea3307ea55f4c2f96b12f5883d28751" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "39de4f0f5422417e886051530c80aa17"
I20260809 15:35:58.343629 8834 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "916879440a494083be11fd8782090277" candidate_uuid: "8ea3307ea55f4c2f96b12f5883d28751" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cfe93bb4592d455c87f61dfd5ca87ddb"
I20260809 15:35:58.343796 8694 raft_consensus.cc:3037] T 916879440a494083be11fd8782090277 P 39de4f0f5422417e886051530c80aa17 [term 0 FOLLOWER]: Advancing to term 1
I20260809 15:35:58.343842 8834 raft_consensus.cc:3037] T 916879440a494083be11fd8782090277 P cfe93bb4592d455c87f61dfd5ca87ddb [term 0 FOLLOWER]: Advancing to term 1
I20260809 15:35:58.345041 8694 raft_consensus.cc:2454] T 916879440a494083be11fd8782090277 P 39de4f0f5422417e886051530c80aa17 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8ea3307ea55f4c2f96b12f5883d28751 in term 1.
I20260809 15:35:58.345494 8487 leader_election.cc:302] T 916879440a494083be11fd8782090277 P 8ea3307ea55f4c2f96b12f5883d28751 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 39de4f0f5422417e886051530c80aa17, 8ea3307ea55f4c2f96b12f5883d28751; no voters:
I20260809 15:35:58.345649 8834 raft_consensus.cc:2454] T 916879440a494083be11fd8782090277 P cfe93bb4592d455c87f61dfd5ca87ddb [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8ea3307ea55f4c2f96b12f5883d28751 in term 1.
I20260809 15:35:58.345692 8911 raft_consensus.cc:2781] T 916879440a494083be11fd8782090277 P 8ea3307ea55f4c2f96b12f5883d28751 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 15:35:58.345973 8911 raft_consensus.cc:686] T 916879440a494083be11fd8782090277 P 8ea3307ea55f4c2f96b12f5883d28751 [term 1 LEADER]: Becoming Leader. State: Replica: 8ea3307ea55f4c2f96b12f5883d28751, State: Running, Role: LEADER
I20260809 15:35:58.346335 8911 consensus_queue.cc:237] T 916879440a494083be11fd8782090277 P 8ea3307ea55f4c2f96b12f5883d28751 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cfe93bb4592d455c87f61dfd5ca87ddb" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41025 } } peers { permanent_uuid: "39de4f0f5422417e886051530c80aa17" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 44053 } } peers { permanent_uuid: "8ea3307ea55f4c2f96b12f5883d28751" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 33727 } }
I20260809 15:35:58.349749 8404 catalog_manager.cc:5495] T 916879440a494083be11fd8782090277 P 8ea3307ea55f4c2f96b12f5883d28751 reported cstate change: term changed from 0 to 1, leader changed from <none> to 8ea3307ea55f4c2f96b12f5883d28751 (127.3.89.193). New cstate: current_term: 1 leader_uuid: "8ea3307ea55f4c2f96b12f5883d28751" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cfe93bb4592d455c87f61dfd5ca87ddb" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41025 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "39de4f0f5422417e886051530c80aa17" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 44053 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8ea3307ea55f4c2f96b12f5883d28751" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 33727 } health_report { overall_health: HEALTHY } } }
I20260809 15:35:58.365113 8901 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:35:58.363389 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57276 (local address 127.3.89.193:33727)
0809 15:35:58.363540 (+ 151us) server_negotiation.cc:206] Beginning negotiation
0809 15:35:58.363546 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 15:35:58.363561 (+ 15us) server_negotiation.cc:407] Connection header received
0809 15:35:58.363763 (+ 202us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 15:35:58.363766 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 15:35:58.363826 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 15:35:58.363924 (+ 98us) server_negotiation.cc:226] Negotiated authn=SASL
0809 15:35:58.364696 (+ 772us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 15:35:58.364700 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 15:35:58.364702 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 15:35:58.364714 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0809 15:35:58.364742 (+ 28us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 15:35:58.364750 (+ 8us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 15:35:58.364844 (+ 94us) server_negotiation.cc:1035] Waiting for connection context
0809 15:35:58.364918 (+ 74us) server_negotiation.cc:299] Negotiation successful
0809 15:35:58.364968 (+ 50us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":60}
I20260809 15:35:58.370921 8694 raft_consensus.cc:1261] T 916879440a494083be11fd8782090277 P 39de4f0f5422417e886051530c80aa17 [term 1 FOLLOWER]: Refusing update from remote peer 8ea3307ea55f4c2f96b12f5883d28751: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 15:35:58.370921 8834 raft_consensus.cc:1261] T 916879440a494083be11fd8782090277 P cfe93bb4592d455c87f61dfd5ca87ddb [term 1 FOLLOWER]: Refusing update from remote peer 8ea3307ea55f4c2f96b12f5883d28751: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 15:35:58.371765 8914 consensus_queue.cc:1035] T 916879440a494083be11fd8782090277 P 8ea3307ea55f4c2f96b12f5883d28751 [LEADER]: Connected to new peer: Peer: permanent_uuid: "39de4f0f5422417e886051530c80aa17" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 44053 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 15:35:58.371953 8911 consensus_queue.cc:1035] T 916879440a494083be11fd8782090277 P 8ea3307ea55f4c2f96b12f5883d28751 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cfe93bb4592d455c87f61dfd5ca87ddb" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41025 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 15:35:58.379607 8918 mvcc.cc:205] Tried to move back new op lower bound from 7316642850279952384 to 7316642850189512704. Current Snapshot: MvccSnapshot[applied={T|T < 7316642850279952384}]
I20260809 15:35:58.381342 8919 mvcc.cc:205] Tried to move back new op lower bound from 7316642850279952384 to 7316642850189512704. Current Snapshot: MvccSnapshot[applied={T|T < 7316642850279952384}]
I20260809 15:35:58.387797 8404 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:51112:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 15:35:58.388003 8404 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:51112:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 15:35:58.390692 8404 catalog_manager.cc:5782] T 00000000000000000000000000000000 P da58c50a461a416a83f1a0e86f316b77: Sending DeleteTablet for 3 replicas of tablet 916879440a494083be11fd8782090277
I20260809 15:35:58.391546 8814 tablet_service.cc:1514] Processing DeleteTablet for tablet 916879440a494083be11fd8782090277 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 15:35:58 UTC) from {username='slave'} at 127.0.0.1:36544
I20260809 15:35:58.391530 8673 tablet_service.cc:1514] Processing DeleteTablet for tablet 916879440a494083be11fd8782090277 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 15:35:58 UTC) from {username='slave'} at 127.0.0.1:55014
I20260809 15:35:58.392071 8928 tablet_replica.cc:330] T 916879440a494083be11fd8782090277 P 39de4f0f5422417e886051530c80aa17: stopping tablet replica
I20260809 15:35:58.392231 8535 tablet_service.cc:1514] Processing DeleteTablet for tablet 916879440a494083be11fd8782090277 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 15:35:58 UTC) from {username='slave'} at 127.0.0.1:33848
I20260809 15:35:58.392429 8928 raft_consensus.cc:2229] T 916879440a494083be11fd8782090277 P 39de4f0f5422417e886051530c80aa17 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 15:35:58.392753 8928 raft_consensus.cc:2258] T 916879440a494083be11fd8782090277 P 39de4f0f5422417e886051530c80aa17 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 15:35:58.392787 8927 tablet_replica.cc:330] T 916879440a494083be11fd8782090277 P cfe93bb4592d455c87f61dfd5ca87ddb: stopping tablet replica
I20260809 15:35:58.392997 3431 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskm314YR/build/debug/bin/kudu with pid 8468
I20260809 15:35:58.396274 8928 ts_tablet_manager.cc:1905] T 916879440a494083be11fd8782090277 P 39de4f0f5422417e886051530c80aa17: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 15:35:58.393096 8927 raft_consensus.cc:2229] T 916879440a494083be11fd8782090277 P cfe93bb4592d455c87f61dfd5ca87ddb [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 15:35:58.398072 8927 raft_consensus.cc:2258] T 916879440a494083be11fd8782090277 P cfe93bb4592d455c87f61dfd5ca87ddb [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 15:35:58.399471 8927 ts_tablet_manager.cc:1905] T 916879440a494083be11fd8782090277 P cfe93bb4592d455c87f61dfd5ca87ddb: Deleting tablet data with delete state TABLET_DATA_DELETED
W20260809 15:35:58.403890 8389 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.3.89.193:33727 (error 108)
I20260809 15:35:58.404804 3431 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskm314YR/build/debug/bin/kudu with pid 8606
I20260809 15:35:58.404954 8897 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:35:58.404649 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.89.193:33727 (local address 127.0.0.1:57278)
0809 15:35:58.404773 (+ 124us) negotiation.cc:107] Waiting for socket to connect
0809 15:35:58.404849 (+ 76us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.89.193:33727: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":29}
W20260809 15:35:58.405790 8389 catalog_manager.cc:4584] TS 8ea3307ea55f4c2f96b12f5883d28751 (127.3.89.193:33727): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 916879440a494083be11fd8782090277: Network error: Client connection negotiation failed: client connection to 127.3.89.193:33727: connect: Connection refused (error 111)
I20260809 15:35:58.446635 8927 ts_tablet_manager.cc:1918] T 916879440a494083be11fd8782090277 P cfe93bb4592d455c87f61dfd5ca87ddb: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 15:35:58.446817 8927 log.cc:1196] T 916879440a494083be11fd8782090277 P cfe93bb4592d455c87f61dfd5ca87ddb: Deleting WAL directory at /tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786289732823074-3431-0/minicluster-data/ts-2/wal/wals/916879440a494083be11fd8782090277
I20260809 15:35:58.447297 8927 ts_tablet_manager.cc:1939] T 916879440a494083be11fd8782090277 P cfe93bb4592d455c87f61dfd5ca87ddb: Deleting consensus metadata
I20260809 15:35:58.449316 3431 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskm314YR/build/debug/bin/kudu with pid 8745
I20260809 15:35:58.453157 8897 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:35:58.449680 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.89.194:44053 (local address 127.0.0.1:54534)
0809 15:35:58.453014 (+ 3334us) negotiation.cc:107] Waiting for socket to connect
0809 15:35:58.453070 (+ 56us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.89.194:44053: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":3237}
I20260809 15:35:58.458822 8897 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:35:58.458536 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.89.193:33727 (local address 127.0.0.1:57282)
0809 15:35:58.458670 (+ 134us) negotiation.cc:107] Waiting for socket to connect
0809 15:35:58.458723 (+ 53us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.89.193:33727: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":46}
I20260809 15:35:58.459502 8897 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:35:58.459315 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.89.193:33727 (local address 127.0.0.1:57288)
0809 15:35:58.459408 (+ 93us) negotiation.cc:107] Waiting for socket to connect
0809 15:35:58.459451 (+ 43us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.89.193:33727: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":36}
I20260809 15:35:58.475777 3431 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskm314YR/build/debug/bin/kudu with pid 8374
2026-08-09T15:35:58Z chronyd exiting
[ OK ] SecurityITest.TestDisableAuthenticationEncryption (1100 ms)
[ RUN ] SecurityITest.TestJwtMiniCluster
I20260809 15:35:58.500299 3431 mini_oidc.cc:152] Starting JWKS server
I20260809 15:35:58.502053 3431 webserver.cc:460] Webserver started at https://127.0.0.1:38753/ using document root <none> and password file <none>
I20260809 15:35:58.502233 3431 mini_oidc.cc:189] Starting OIDC Discovery server
I20260809 15:35:58.502537 3431 webserver.cc:460] Webserver started at http://127.0.0.1:44891/ using document root <none> and password file <none>
2026-08-09T15:35:58Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T15:35:58Z Disabled control of system clock
I20260809 15:35:58.512888 3431 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskm314YR/build/debug/bin/kudu
/tmp/dist-test-taskm314YR/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786289732823074-3431-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786289732823074-3431-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786289732823074-3431-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786289732823074-3431-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.3.89.254:33409
--webserver_interface=127.3.89.254
--webserver_port=0
--builtin_ntp_servers=127.3.89.212:33023
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.3.89.254:33409
--enable_jwt_token_auth=true
--jwks_url=https://localhost:38753/jwks.json
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--trusted_certificate_file=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786289732823074-3431-0/testchainca.cert with env {}
W20260809 15:35:58.629900 8937 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 15:35:58.630160 8937 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 15:35:58.630200 8937 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 15:35:58.634298 8937 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 15:35:58.634363 8937 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 15:35:58.634379 8937 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 15:35:58.634397 8937 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 15:35:58.634411 8937 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
W20260809 15:35:58.634428 8937 flags.cc:425] Enabled experimental flag: --enable_jwt_token_auth=true
W20260809 15:35:58.634446 8937 flags.cc:425] Enabled experimental flag: --jwks_url=https://localhost:38753/jwks.json
I20260809 15:35:58.640615 8937 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.3.89.212:33023
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786289732823074-3431-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786289732823074-3431-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.3.89.254:33409
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.3.89.254:33409
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--enable_jwt_token_auth=true
--jwks_url=https://localhost:38753/jwks.json
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786289732823074-3431-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.3.89.254
--webserver_port=0
--trusted_certificate_file=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786289732823074-3431-0/testchainca.cert
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.8937
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786289732823074-3431-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 09 Aug 2026 14:46:55 UTC on ccycloud.quasar-yswkmf.root.comops.site
build id 4576540
I20260809 15:35:58.641645 8937 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 15:35:58.642611 8937 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260809 15:35:58.651475 8937 server_base.cc:1034] running on GCE node
W20260809 15:35:58.651499 8942 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 15:35:58.651691 8946 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 15:35:58.651525 8943 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 15:35:58.652366 8937 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 15:35:58.653501 8937 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 15:35:58.654690 8937 hybrid_clock.cc:648] HybridClock initialized: now 1786289758654673 us; error 37 us; skew 500 ppm
I20260809 15:35:58.656670 8937 webserver.cc:460] Webserver started at http://127.3.89.254:41915/ using document root <none> and password file <none>
I20260809 15:35:58.657275 8937 fs_manager.cc:359] Metadata directory not provided
I20260809 15:35:58.657374 8937 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 15:35:58.657610 8937 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 15:35:58.659709 8937 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786289732823074-3431-0/minicluster-data/master-0/data/instance:
uuid: "ab9804c6f85544afaec86eef25be33f2"
format_stamp: "Formatted at 2026-08-09 15:35:58 on dist-test-slave-2w3w"
server_key: "76dcc702b42f27f4f93c15f3975bb8c8"
server_key_iv: "dce24b36720da3c37242088ac27cae29"
server_key_version: "encryptionkey@0"
I20260809 15:35:58.660295 8937 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786289732823074-3431-0/minicluster-data/master-0/wal/instance:
uuid: "ab9804c6f85544afaec86eef25be33f2"
format_stamp: "Formatted at 2026-08-09 15:35:58 on dist-test-slave-2w3w"
server_key: "76dcc702b42f27f4f93c15f3975bb8c8"
server_key_iv: "dce24b36720da3c37242088ac27cae29"
server_key_version: "encryptionkey@0"
I20260809 15:35:58.663975 8937 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.004s sys 0.000s
I20260809 15:35:58.666292 8952 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 15:35:58.667572 8937 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260809 15:35:58.667721 8937 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786289732823074-3431-0/minicluster-data/master-0/data,/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786289732823074-3431-0/minicluster-data/master-0/wal
uuid: "ab9804c6f85544afaec86eef25be33f2"
format_stamp: "Formatted at 2026-08-09 15:35:58 on dist-test-slave-2w3w"
server_key: "76dcc702b42f27f4f93c15f3975bb8c8"
server_key_iv: "dce24b36720da3c37242088ac27cae29"
server_key_version: "encryptionkey@0"
I20260809 15:35:58.667834 8937 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786289732823074-3431-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786289732823074-3431-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786289732823074-3431-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 15:35:58.684199 8937 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 15:35:58.684913 8937 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 15:35:58.685122 8937 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 15:35:58.692867 8937 rpc_server.cc:307] RPC server started. Bound to: 127.3.89.254:33409
I20260809 15:35:58.692950 9011 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.3.89.254:33409 every 8 connection(s)
I20260809 15:35:58.693852 8937 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786289732823074-3431-0/minicluster-data/master-0/data/info.pb
I20260809 15:35:58.697185 9012 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 15:35:58.699144 3431 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskm314YR/build/debug/bin/kudu as pid 8937
I20260809 15:35:58.699288 3431 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786289732823074-3431-0/minicluster-data/master-0/wal/instance
I20260809 15:35:58.699572 3431 external_mini_cluster.cc:1428] Setting key 5cf6ed289e050dded3163fd9bd7192e2
I20260809 15:35:58.703779 9012 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ab9804c6f85544afaec86eef25be33f2: Bootstrap starting.
I20260809 15:35:58.706360 9012 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P ab9804c6f85544afaec86eef25be33f2: Neither blocks nor log segments found. Creating new log.
I20260809 15:35:58.707195 9012 log.cc:824] T 00000000000000000000000000000000 P ab9804c6f85544afaec86eef25be33f2: Log is configured to *not* fsync() on all Append() calls
I20260809 15:35:58.709345 9012 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ab9804c6f85544afaec86eef25be33f2: No bootstrap required, opened a new log
I20260809 15:35:58.710862 9015 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:35:58.700937 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40806 (local address 127.3.89.254:33409)
0809 15:35:58.701597 (+ 660us) server_negotiation.cc:206] Beginning negotiation
0809 15:35:58.701618 (+ 21us) server_negotiation.cc:399] Waiting for connection header
0809 15:35:58.701660 (+ 42us) server_negotiation.cc:407] Connection header received
0809 15:35:58.702708 (+ 1048us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 15:35:58.702732 (+ 24us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 15:35:58.703243 (+ 511us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 15:35:58.703591 (+ 348us) server_negotiation.cc:226] Negotiated authn=SASL
0809 15:35:58.704043 (+ 452us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:35:58.706336 (+ 2293us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:35:58.707302 (+ 966us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:35:58.707899 (+ 597us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:35:58.708106 (+ 207us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:35:58.708873 (+ 767us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 15:35:58.708879 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 15:35:58.708894 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 15:35:58.709374 (+ 480us) server_negotiation.cc:922] calling sasl_server_start()
0809 15:35:58.709455 (+ 81us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 15:35:58.709482 (+ 27us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 15:35:58.709593 (+ 111us) server_negotiation.cc:1035] Waiting for connection context
0809 15:35:58.709717 (+ 124us) server_negotiation.cc:299] Negotiation successful
0809 15:35:58.709980 (+ 263us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":266,"thread_start_us":104,"threads_started":1}
I20260809 15:35:58.713728 3431 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260809 15:35:58.713757 9012 raft_consensus.cc:348] T 00000000000000000000000000000000 P ab9804c6f85544afaec86eef25be33f2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ab9804c6f85544afaec86eef25be33f2" member_type: VOTER last_known_addr { host: "127.3.89.254" port: 33409 } }
I20260809 15:35:58.714041 9012 raft_consensus.cc:374] T 00000000000000000000000000000000 P ab9804c6f85544afaec86eef25be33f2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 15:35:58.714109 9012 raft_consensus.cc:729] T 00000000000000000000000000000000 P ab9804c6f85544afaec86eef25be33f2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ab9804c6f85544afaec86eef25be33f2, State: Initialized, Role: FOLLOWER
I20260809 15:35:58.714547 9012 consensus_queue.cc:260] T 00000000000000000000000000000000 P ab9804c6f85544afaec86eef25be33f2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ab9804c6f85544afaec86eef25be33f2" member_type: VOTER last_known_addr { host: "127.3.89.254" port: 33409 } }
I20260809 15:35:58.714697 9012 raft_consensus.cc:388] T 00000000000000000000000000000000 P ab9804c6f85544afaec86eef25be33f2 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 15:35:58.714784 9012 raft_consensus.cc:482] T 00000000000000000000000000000000 P ab9804c6f85544afaec86eef25be33f2 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 15:35:58.714884 9012 raft_consensus.cc:3037] T 00000000000000000000000000000000 P ab9804c6f85544afaec86eef25be33f2 [term 0 FOLLOWER]: Advancing to term 1
I20260809 15:35:58.756760 9012 raft_consensus.cc:504] T 00000000000000000000000000000000 P ab9804c6f85544afaec86eef25be33f2 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ab9804c6f85544afaec86eef25be33f2" member_type: VOTER last_known_addr { host: "127.3.89.254" port: 33409 } }
I20260809 15:35:58.757336 9012 leader_election.cc:302] T 00000000000000000000000000000000 P ab9804c6f85544afaec86eef25be33f2 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: ab9804c6f85544afaec86eef25be33f2; no voters:
I20260809 15:35:58.757802 9012 leader_election.cc:288] T 00000000000000000000000000000000 P ab9804c6f85544afaec86eef25be33f2 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 15:35:58.757961 9017 raft_consensus.cc:2781] T 00000000000000000000000000000000 P ab9804c6f85544afaec86eef25be33f2 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 15:35:58.758236 9017 raft_consensus.cc:686] T 00000000000000000000000000000000 P ab9804c6f85544afaec86eef25be33f2 [term 1 LEADER]: Becoming Leader. State: Replica: ab9804c6f85544afaec86eef25be33f2, State: Running, Role: LEADER
I20260809 15:35:58.759223 9017 consensus_queue.cc:237] T 00000000000000000000000000000000 P ab9804c6f85544afaec86eef25be33f2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ab9804c6f85544afaec86eef25be33f2" member_type: VOTER last_known_addr { host: "127.3.89.254" port: 33409 } }
I20260809 15:35:58.759526 9012 sys_catalog.cc:564] T 00000000000000000000000000000000 P ab9804c6f85544afaec86eef25be33f2 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 15:35:58.760919 9018 sys_catalog.cc:455] T 00000000000000000000000000000000 P ab9804c6f85544afaec86eef25be33f2 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "ab9804c6f85544afaec86eef25be33f2" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ab9804c6f85544afaec86eef25be33f2" member_type: VOTER last_known_addr { host: "127.3.89.254" port: 33409 } } }
I20260809 15:35:58.761060 9018 sys_catalog.cc:458] T 00000000000000000000000000000000 P ab9804c6f85544afaec86eef25be33f2 [sys.catalog]: This master's current role is: LEADER
I20260809 15:35:58.761490 9019 sys_catalog.cc:455] T 00000000000000000000000000000000 P ab9804c6f85544afaec86eef25be33f2 [sys.catalog]: SysCatalogTable state changed. Reason: New leader ab9804c6f85544afaec86eef25be33f2. Latest consensus state: current_term: 1 leader_uuid: "ab9804c6f85544afaec86eef25be33f2" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ab9804c6f85544afaec86eef25be33f2" member_type: VOTER last_known_addr { host: "127.3.89.254" port: 33409 } } }
I20260809 15:35:58.761648 9019 sys_catalog.cc:458] T 00000000000000000000000000000000 P ab9804c6f85544afaec86eef25be33f2 [sys.catalog]: This master's current role is: LEADER
W20260809 15:35:58.765093 9034 catalog_manager.cc:1558] T 00000000000000000000000000000000 P ab9804c6f85544afaec86eef25be33f2: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260809 15:35:58.765192 9034 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260809 15:35:58.765268 9026 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 15:35:58.765974 9026 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 15:35:58.771370 9026 catalog_manager.cc:1347] Generated new cluster ID: cdaefeaee5634c9096e51d354bf8edc7
I20260809 15:35:58.771436 9026 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 15:35:58.793447 9026 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 15:35:58.794728 9026 catalog_manager.cc:1504] Loading token signing keys...
I20260809 15:35:58.802922 9026 catalog_manager.cc:5868] T 00000000000000000000000000000000 P ab9804c6f85544afaec86eef25be33f2: Generated new TSK 0
I20260809 15:35:58.803740 9026 catalog_manager.cc:1514] Initializing in-progress tserver states...
E20260809 15:35:58.839349 9044 webserver.cc:533] Webserver: sslize failed: error:14094415:SSL routines:ssl3_read_bytes:sslv3 alert certificate expired
E20260809 15:35:58.839507 9044 webserver.cc:533] Webserver: error shutting down SSL connection: error:140E0197:SSL routines:SSL_shutdown:shutdown while in init
I20260809 15:35:58.840119 9015 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:35:58.821942 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40808 (local address 127.3.89.254:33409)
0809 15:35:58.822087 (+ 145us) server_negotiation.cc:206] Beginning negotiation
0809 15:35:58.822091 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 15:35:58.822263 (+ 172us) server_negotiation.cc:407] Connection header received
0809 15:35:58.822501 (+ 238us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 15:35:58.822504 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 15:35:58.822553 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 15:35:58.822644 (+ 91us) server_negotiation.cc:226] Negotiated authn=JWT
0809 15:35:58.822973 (+ 329us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:35:58.823594 (+ 621us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:35:58.824828 (+ 1234us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:35:58.825051 (+ 223us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:35:58.825131 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:35:58.825508 (+ 377us) server_negotiation.cc:365] Received JWT_EXCHANGE NegotiatePB request
0809 15:35:58.839777 (+ 14269us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:38753/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
0809 15:35:58.840007 (+ 230us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:40808: Failed to load JWKS: Error downloading JWKS from 'https://localhost:38753/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
Metrics: {"server-negotiator.queue_time_us":54}
W20260809 15:35:58.840224 9015 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:40808: Failed to load JWKS: Error downloading JWKS from 'https://localhost:38753/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
W20260809 15:35:58.841764 9043 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 15:35:58.821768 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.89.254:33409 (local address 127.0.0.1:40808)
0809 15:35:58.822167 (+ 399us) negotiation.cc:107] Waiting for socket to connect
0809 15:35:58.822183 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0809 15:35:58.822318 (+ 135us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 15:35:58.822682 (+ 364us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 15:35:58.822686 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 15:35:58.822699 (+ 13us) client_negotiation.cc:189] Negotiated authn=JWT
0809 15:35:58.822838 (+ 139us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:35:58.822845 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:35:58.823723 (+ 878us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:35:58.823726 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:35:58.824706 (+ 980us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:35:58.824724 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:35:58.825167 (+ 443us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:35:58.825179 (+ 12us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:35:58.825277 (+ 98us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:35:58.825329 (+ 52us) client_negotiation.cc:252] Sending JWT_EXCHANGE NegotiatePB request
0809 15:35:58.839998 (+ 14669us) client_negotiation.cc:283] Received error response from server: Runtime error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:38753/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
0809 15:35:58.840263 (+ 265us) negotiation.cc:327] Negotiation complete: Runtime error: Client connection negotiation failed: client connection to 127.3.89.254:33409: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:38753/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
Metrics: {"client-negotiator.queue_time_us":268,"thread_start_us":153,"threads_started":1}
/data0/somelongdirectorytoavoidrpathissues/src/kudu/src/kudu/integration-tests/security-itest.cc:597: Failure
Failed
Bad status: Runtime error: Could not connect to the cluster: Client connection negotiation failed: client connection to 127.3.89.254:33409: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:38753/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
Google Test trace:
/data0/somelongdirectorytoavoidrpathissues/src/kudu/src/kudu/integration-tests/security-itest.cc:593: Valid JWT
I20260809 15:35:58.842537 3431 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskm314YR/build/debug/bin/kudu with pid 8937
2026-08-09T15:35:58Z chronyd exiting
I20260809 15:35:58.882483 3431 test_util.cc:175] -----------------------------------------------
I20260809 15:35:58.882570 3431 test_util.cc:176] Had fatal failures, leaving test files at /tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786289732823074-3431-0
[ FAILED ] SecurityITest.TestJwtMiniCluster (384 ms)
[ RUN ] SecurityITest.TestJwtMiniClusterWithInvalidCert
I20260809 15:35:58.883862 3431 mini_oidc.cc:152] Starting JWKS server
I20260809 15:35:58.884747 3431 webserver.cc:460] Webserver started at https://127.0.0.1:40457/ using document root <none> and password file <none>
I20260809 15:35:58.884833 3431 mini_oidc.cc:189] Starting OIDC Discovery server
I20260809 15:35:58.885043 3431 webserver.cc:460] Webserver started at http://127.0.0.1:39261/ using document root <none> and password file <none>
2026-08-09T15:35:58Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T15:35:58Z Disabled control of system clock
I20260809 15:35:58.971798 3431 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskm314YR/build/debug/bin/kudu
/tmp/dist-test-taskm314YR/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786289732823074-3431-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786289732823074-3431-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786289732823074-3431-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786289732823074-3431-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.3.89.254:41779
--webserver_interface=127.3.89.254
--webserver_port=0
--builtin_ntp_servers=127.3.89.212:35687
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.3.89.254:41779
--enable_jwt_token_auth=true
--jwks_url=https://localhost:40457/jwks.json
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--trusted_certificate_file=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786289732823074-3431-0/testchainca.cert with env {}
W20260809 15:35:59.092978 9051 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 15:35:59.093300 9051 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 15:35:59.093384 9051 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 15:35:59.097762 9051 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 15:35:59.097863 9051 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 15:35:59.097892 9051 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 15:35:59.097944 9051 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 15:35:59.097986 9051 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
W20260809 15:35:59.098017 9051 flags.cc:425] Enabled experimental flag: --enable_jwt_token_auth=true
W20260809 15:35:59.098062 9051 flags.cc:425] Enabled experimental flag: --jwks_url=https://localhost:40457/jwks.json
I20260809 15:35:59.104174 9051 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.3.89.212:35687
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786289732823074-3431-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786289732823074-3431-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.3.89.254:41779
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.3.89.254:41779
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--enable_jwt_token_auth=true
--jwks_url=https://localhost:40457/jwks.json
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786289732823074-3431-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.3.89.254
--webserver_port=0
--trusted_certificate_file=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786289732823074-3431-0/testchainca.cert
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.9051
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786289732823074-3431-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 09 Aug 2026 14:46:55 UTC on ccycloud.quasar-yswkmf.root.comops.site
build id 4576540
I20260809 15:35:59.105340 9051 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 15:35:59.106361 9051 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 15:35:59.115036 9057 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 15:35:59.115113 9059 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 15:35:59.115182 9051 server_base.cc:1034] running on GCE node
W20260809 15:35:59.115098 9056 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 15:35:59.115829 9051 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 15:35:59.116953 9051 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 15:35:59.118136 9051 hybrid_clock.cc:648] HybridClock initialized: now 1786289759118112 us; error 45 us; skew 500 ppm
I20260809 15:35:59.124015 9051 webserver.cc:460] Webserver started at http://127.3.89.254:34485/ using document root <none> and password file <none>
I20260809 15:35:59.124639 9051 fs_manager.cc:359] Metadata directory not provided
I20260809 15:35:59.124768 9051 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 15:35:59.125039 9051 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 15:35:59.127449 9051 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786289732823074-3431-0/minicluster-data/master-0/data/instance:
uuid: "043e54c8879e453f96b88a63738018a9"
format_stamp: "Formatted at 2026-08-09 15:35:59 on dist-test-slave-2w3w"
server_key: "a7addc762648dfb8187c4c67464f972d"
server_key_iv: "7243a8612ec15174019a406d8c2f25c1"
server_key_version: "encryptionkey@0"
I20260809 15:35:59.128024 9051 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786289732823074-3431-0/minicluster-data/master-0/wal/instance:
uuid: "043e54c8879e453f96b88a63738018a9"
format_stamp: "Formatted at 2026-08-09 15:35:59 on dist-test-slave-2w3w"
server_key: "a7addc762648dfb8187c4c67464f972d"
server_key_iv: "7243a8612ec15174019a406d8c2f25c1"
server_key_version: "encryptionkey@0"
I20260809 15:35:59.131743 9051 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.003s sys 0.001s
I20260809 15:35:59.134168 9067 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 15:35:59.135478 9051 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260809 15:35:59.135622 9051 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786289732823074-3431-0/minicluster-data/master-0/data,/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786289732823074-3431-0/minicluster-data/master-0/wal
uuid: "043e54c8879e453f96b88a63738018a9"
format_stamp: "Formatted at 2026-08-09 15:35:59 on dist-test-slave-2w3w"
server_key: "a7addc762648dfb8187c4c67464f972d"
server_key_iv: "7243a8612ec15174019a406d8c2f25c1"
server_key_version: "encryptionkey@0"
I20260809 15:35:59.135736 9051 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786289732823074-3431-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786289732823074-3431-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786289732823074-3431-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 15:35:59.160519 9051 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 15:35:59.161337 9051 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 15:35:59.161550 9051 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 15:35:59.169308 9051 rpc_server.cc:307] RPC server started. Bound to: 127.3.89.254:41779
I20260809 15:35:59.169322 9121 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.3.89.254:41779 every 8 connection(s)
I20260809 15:35:59.170454 9051 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786289732823074-3431-0/minicluster-data/master-0/data/info.pb
I20260809 15:35:59.173570 9122 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 15:35:59.178269 3431 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskm314YR/build/debug/bin/kudu as pid 9051
I20260809 15:35:59.178381 3431 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786289732823074-3431-0/minicluster-data/master-0/wal/instance
I20260809 15:35:59.178694 3431 external_mini_cluster.cc:1428] Setting key 8d87f65c0c62f5923256664d6c65bd07
I20260809 15:35:59.179422 9122 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 043e54c8879e453f96b88a63738018a9: Bootstrap starting.
I20260809 15:35:59.182058 9122 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 043e54c8879e453f96b88a63738018a9: Neither blocks nor log segments found. Creating new log.
I20260809 15:35:59.183009 9122 log.cc:824] T 00000000000000000000000000000000 P 043e54c8879e453f96b88a63738018a9: Log is configured to *not* fsync() on all Append() calls
I20260809 15:35:59.185194 9122 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 043e54c8879e453f96b88a63738018a9: No bootstrap required, opened a new log
I20260809 15:35:59.189554 9127 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:35:59.180164 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50676 (local address 127.3.89.254:41779)
0809 15:35:59.180885 (+ 721us) server_negotiation.cc:206] Beginning negotiation
0809 15:35:59.180908 (+ 23us) server_negotiation.cc:399] Waiting for connection header
0809 15:35:59.180952 (+ 44us) server_negotiation.cc:407] Connection header received
0809 15:35:59.181785 (+ 833us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 15:35:59.181808 (+ 23us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 15:35:59.182327 (+ 519us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 15:35:59.182686 (+ 359us) server_negotiation.cc:226] Negotiated authn=SASL
0809 15:35:59.183185 (+ 499us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:35:59.184551 (+ 1366us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:35:59.185546 (+ 995us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:35:59.186267 (+ 721us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:35:59.186428 (+ 161us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:35:59.187323 (+ 895us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 15:35:59.187330 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 15:35:59.187348 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 15:35:59.187858 (+ 510us) server_negotiation.cc:922] calling sasl_server_start()
0809 15:35:59.187929 (+ 71us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 15:35:59.187955 (+ 26us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 15:35:59.188068 (+ 113us) server_negotiation.cc:1035] Waiting for connection context
0809 15:35:59.188176 (+ 108us) server_negotiation.cc:299] Negotiation successful
0809 15:35:59.188459 (+ 283us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":291,"thread_start_us":103,"threads_started":1}
I20260809 15:35:59.190107 9122 raft_consensus.cc:348] T 00000000000000000000000000000000 P 043e54c8879e453f96b88a63738018a9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "043e54c8879e453f96b88a63738018a9" member_type: VOTER last_known_addr { host: "127.3.89.254" port: 41779 } }
I20260809 15:35:59.190323 9122 raft_consensus.cc:374] T 00000000000000000000000000000000 P 043e54c8879e453f96b88a63738018a9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 15:35:59.190382 9122 raft_consensus.cc:729] T 00000000000000000000000000000000 P 043e54c8879e453f96b88a63738018a9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 043e54c8879e453f96b88a63738018a9, State: Initialized, Role: FOLLOWER
I20260809 15:35:59.190851 9122 consensus_queue.cc:260] T 00000000000000000000000000000000 P 043e54c8879e453f96b88a63738018a9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "043e54c8879e453f96b88a63738018a9" member_type: VOTER last_known_addr { host: "127.3.89.254" port: 41779 } }
I20260809 15:35:59.191030 9122 raft_consensus.cc:388] T 00000000000000000000000000000000 P 043e54c8879e453f96b88a63738018a9 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 15:35:59.191126 9122 raft_consensus.cc:482] T 00000000000000000000000000000000 P 043e54c8879e453f96b88a63738018a9 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 15:35:59.191228 9122 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 043e54c8879e453f96b88a63738018a9 [term 0 FOLLOWER]: Advancing to term 1
I20260809 15:35:59.192512 3431 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260809 15:35:59.274641 9122 raft_consensus.cc:504] T 00000000000000000000000000000000 P 043e54c8879e453f96b88a63738018a9 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "043e54c8879e453f96b88a63738018a9" member_type: VOTER last_known_addr { host: "127.3.89.254" port: 41779 } }
I20260809 15:35:59.275213 9122 leader_election.cc:302] T 00000000000000000000000000000000 P 043e54c8879e453f96b88a63738018a9 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 043e54c8879e453f96b88a63738018a9; no voters:
I20260809 15:35:59.275615 9122 leader_election.cc:288] T 00000000000000000000000000000000 P 043e54c8879e453f96b88a63738018a9 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 15:35:59.275763 9129 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 043e54c8879e453f96b88a63738018a9 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 15:35:59.276072 9129 raft_consensus.cc:686] T 00000000000000000000000000000000 P 043e54c8879e453f96b88a63738018a9 [term 1 LEADER]: Becoming Leader. State: Replica: 043e54c8879e453f96b88a63738018a9, State: Running, Role: LEADER
I20260809 15:35:59.276508 9129 consensus_queue.cc:237] T 00000000000000000000000000000000 P 043e54c8879e453f96b88a63738018a9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "043e54c8879e453f96b88a63738018a9" member_type: VOTER last_known_addr { host: "127.3.89.254" port: 41779 } }
I20260809 15:35:59.277057 9122 sys_catalog.cc:564] T 00000000000000000000000000000000 P 043e54c8879e453f96b88a63738018a9 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 15:35:59.278291 9130 sys_catalog.cc:455] T 00000000000000000000000000000000 P 043e54c8879e453f96b88a63738018a9 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "043e54c8879e453f96b88a63738018a9" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "043e54c8879e453f96b88a63738018a9" member_type: VOTER last_known_addr { host: "127.3.89.254" port: 41779 } } }
I20260809 15:35:59.278446 9130 sys_catalog.cc:458] T 00000000000000000000000000000000 P 043e54c8879e453f96b88a63738018a9 [sys.catalog]: This master's current role is: LEADER
I20260809 15:35:59.278824 9131 sys_catalog.cc:455] T 00000000000000000000000000000000 P 043e54c8879e453f96b88a63738018a9 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 043e54c8879e453f96b88a63738018a9. Latest consensus state: current_term: 1 leader_uuid: "043e54c8879e453f96b88a63738018a9" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "043e54c8879e453f96b88a63738018a9" member_type: VOTER last_known_addr { host: "127.3.89.254" port: 41779 } } }
I20260809 15:35:59.279049 9131 sys_catalog.cc:458] T 00000000000000000000000000000000 P 043e54c8879e453f96b88a63738018a9 [sys.catalog]: This master's current role is: LEADER
W20260809 15:35:59.282084 9145 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 043e54c8879e453f96b88a63738018a9: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260809 15:35:59.282188 9145 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260809 15:35:59.282289 9138 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 15:35:59.283093 9138 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 15:35:59.288729 9138 catalog_manager.cc:1347] Generated new cluster ID: 4afae6a7dea344deacf8918b7dd3414b
I20260809 15:35:59.288800 9138 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 15:35:59.297082 9138 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 15:35:59.298300 9138 catalog_manager.cc:1504] Loading token signing keys...
I20260809 15:35:59.311015 9138 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 043e54c8879e453f96b88a63738018a9: Generated new TSK 0
I20260809 15:35:59.311769 9138 catalog_manager.cc:1514] Initializing in-progress tserver states...
E20260809 15:35:59.348762 9158 webserver.cc:533] Webserver: sslize failed: error:14094418:SSL routines:ssl3_read_bytes:tlsv1 alert unknown ca
E20260809 15:35:59.348860 9158 webserver.cc:533] Webserver: error shutting down SSL connection: error:140E0197:SSL routines:SSL_shutdown:shutdown while in init
I20260809 15:35:59.349591 9127 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:35:59.325612 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50680 (local address 127.3.89.254:41779)
0809 15:35:59.325770 (+ 158us) server_negotiation.cc:206] Beginning negotiation
0809 15:35:59.325777 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 15:35:59.326013 (+ 236us) server_negotiation.cc:407] Connection header received
0809 15:35:59.326217 (+ 204us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 15:35:59.326220 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 15:35:59.326274 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 15:35:59.326404 (+ 130us) server_negotiation.cc:226] Negotiated authn=JWT
0809 15:35:59.327435 (+ 1031us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:35:59.327997 (+ 562us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:35:59.328922 (+ 925us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:35:59.329151 (+ 229us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:35:59.329260 (+ 109us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:35:59.329588 (+ 328us) server_negotiation.cc:365] Received JWT_EXCHANGE NegotiatePB request
0809 15:35:59.349118 (+ 19530us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:40457/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
0809 15:35:59.349418 (+ 300us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:50680: Failed to load JWKS: Error downloading JWKS from 'https://localhost:40457/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
Metrics: {"server-negotiator.queue_time_us":67}
W20260809 15:35:59.349681 9127 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:50680: Failed to load JWKS: Error downloading JWKS from 'https://localhost:40457/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
W20260809 15:35:59.349733 9156 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 15:35:59.325538 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.89.254:41779 (local address 127.0.0.1:50680)
0809 15:35:59.325907 (+ 369us) negotiation.cc:107] Waiting for socket to connect
0809 15:35:59.325919 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0809 15:35:59.326058 (+ 139us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 15:35:59.327196 (+ 1138us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 15:35:59.327199 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 15:35:59.327206 (+ 7us) client_negotiation.cc:189] Negotiated authn=JWT
0809 15:35:59.327306 (+ 100us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:35:59.327311 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:35:59.328120 (+ 809us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:35:59.328124 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:35:59.328791 (+ 667us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:35:59.328799 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:35:59.329278 (+ 479us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:35:59.329282 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:35:59.329372 (+ 90us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:35:59.329395 (+ 23us) client_negotiation.cc:252] Sending JWT_EXCHANGE NegotiatePB request
0809 15:35:59.349336 (+ 19941us) client_negotiation.cc:283] Received error response from server: Runtime error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:40457/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
0809 15:35:59.349520 (+ 184us) negotiation.cc:327] Negotiation complete: Runtime error: Client connection negotiation failed: client connection to 127.3.89.254:41779: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:40457/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
Metrics: {"client-negotiator.queue_time_us":304,"thread_start_us":113,"threads_started":1}
I20260809 15:35:59.350373 3431 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskm314YR/build/debug/bin/kudu with pid 9051
2026-08-09T15:35:59Z chronyd exiting
[ OK ] SecurityITest.TestJwtMiniClusterWithInvalidCert (509 ms)
[ RUN ] SecurityITest.TestJwtMiniClusterWithUntrustedCert
I20260809 15:35:59.392968 3431 mini_oidc.cc:152] Starting JWKS server
I20260809 15:35:59.393812 3431 webserver.cc:460] Webserver started at https://127.0.0.1:35853/ using document root <none> and password file <none>
I20260809 15:35:59.393896 3431 mini_oidc.cc:189] Starting OIDC Discovery server
I20260809 15:35:59.394163 3431 webserver.cc:460] Webserver started at http://127.0.0.1:40805/ using document root <none> and password file <none>
2026-08-09T15:35:59Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T15:35:59Z Disabled control of system clock
I20260809 15:35:59.404268 3431 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskm314YR/build/debug/bin/kudu
/tmp/dist-test-taskm314YR/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786289732823074-3431-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786289732823074-3431-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786289732823074-3431-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786289732823074-3431-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.3.89.254:35595
--webserver_interface=127.3.89.254
--webserver_port=0
--builtin_ntp_servers=127.3.89.212:45693
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.3.89.254:35595
--enable_jwt_token_auth=true
--jwks_url=https://localhost:35853/jwks.json
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {}
W20260809 15:35:59.514631 9164 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 15:35:59.515022 9164 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 15:35:59.515111 9164 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 15:35:59.519119 9164 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 15:35:59.519212 9164 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 15:35:59.519241 9164 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 15:35:59.519282 9164 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 15:35:59.519318 9164 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
W20260809 15:35:59.519356 9164 flags.cc:425] Enabled experimental flag: --enable_jwt_token_auth=true
W20260809 15:35:59.519388 9164 flags.cc:425] Enabled experimental flag: --jwks_url=https://localhost:35853/jwks.json
I20260809 15:35:59.524262 9164 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.3.89.212:45693
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786289732823074-3431-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786289732823074-3431-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.3.89.254:35595
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.3.89.254:35595
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--enable_jwt_token_auth=true
--jwks_url=https://localhost:35853/jwks.json
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786289732823074-3431-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.3.89.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.9164
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786289732823074-3431-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 09 Aug 2026 14:46:55 UTC on ccycloud.quasar-yswkmf.root.comops.site
build id 4576540
I20260809 15:35:59.525409 9164 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 15:35:59.526820 9164 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 15:35:59.535204 9169 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 15:35:59.535192 9172 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 15:35:59.535386 9164 server_base.cc:1034] running on GCE node
W20260809 15:35:59.535254 9170 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 15:35:59.536182 9164 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 15:35:59.537326 9164 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 15:35:59.538532 9164 hybrid_clock.cc:648] HybridClock initialized: now 1786289759538503 us; error 40 us; skew 500 ppm
I20260809 15:35:59.540583 9164 webserver.cc:460] Webserver started at http://127.3.89.254:39035/ using document root <none> and password file <none>
I20260809 15:35:59.541158 9164 fs_manager.cc:359] Metadata directory not provided
I20260809 15:35:59.541267 9164 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 15:35:59.541512 9164 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 15:35:59.543757 9164 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786289732823074-3431-0/minicluster-data/master-0/data/instance:
uuid: "d0f8335ba3964bd4bf64ded80c67274d"
format_stamp: "Formatted at 2026-08-09 15:35:59 on dist-test-slave-2w3w"
server_key: "5dd413a24c04134f57993a449e90c30c"
server_key_iv: "63e8cba1491d18f74f57d9c5bdcf1274"
server_key_version: "encryptionkey@0"
I20260809 15:35:59.544341 9164 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786289732823074-3431-0/minicluster-data/master-0/wal/instance:
uuid: "d0f8335ba3964bd4bf64ded80c67274d"
format_stamp: "Formatted at 2026-08-09 15:35:59 on dist-test-slave-2w3w"
server_key: "5dd413a24c04134f57993a449e90c30c"
server_key_iv: "63e8cba1491d18f74f57d9c5bdcf1274"
server_key_version: "encryptionkey@0"
I20260809 15:35:59.548349 9164 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.000s
I20260809 15:35:59.550905 9178 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 15:35:59.552192 9164 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260809 15:35:59.552347 9164 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786289732823074-3431-0/minicluster-data/master-0/data,/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786289732823074-3431-0/minicluster-data/master-0/wal
uuid: "d0f8335ba3964bd4bf64ded80c67274d"
format_stamp: "Formatted at 2026-08-09 15:35:59 on dist-test-slave-2w3w"
server_key: "5dd413a24c04134f57993a449e90c30c"
server_key_iv: "63e8cba1491d18f74f57d9c5bdcf1274"
server_key_version: "encryptionkey@0"
I20260809 15:35:59.552465 9164 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786289732823074-3431-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786289732823074-3431-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786289732823074-3431-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 15:35:59.565243 9164 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 15:35:59.565950 9164 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 15:35:59.566147 9164 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 15:35:59.573747 9164 rpc_server.cc:307] RPC server started. Bound to: 127.3.89.254:35595
I20260809 15:35:59.573786 9235 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.3.89.254:35595 every 8 connection(s)
I20260809 15:35:59.574899 9164 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786289732823074-3431-0/minicluster-data/master-0/data/info.pb
I20260809 15:35:59.578007 9236 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 15:35:59.581072 3431 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskm314YR/build/debug/bin/kudu as pid 9164
I20260809 15:35:59.581178 3431 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786289732823074-3431-0/minicluster-data/master-0/wal/instance
I20260809 15:35:59.581461 3431 external_mini_cluster.cc:1428] Setting key 77fe3988662e39657db3106eb4bae926
I20260809 15:35:59.584197 9236 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d0f8335ba3964bd4bf64ded80c67274d: Bootstrap starting.
I20260809 15:35:59.586632 9236 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P d0f8335ba3964bd4bf64ded80c67274d: Neither blocks nor log segments found. Creating new log.
I20260809 15:35:59.587481 9236 log.cc:824] T 00000000000000000000000000000000 P d0f8335ba3964bd4bf64ded80c67274d: Log is configured to *not* fsync() on all Append() calls
I20260809 15:35:59.589540 9236 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d0f8335ba3964bd4bf64ded80c67274d: No bootstrap required, opened a new log
I20260809 15:35:59.591218 9239 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:35:59.583035 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38196 (local address 127.3.89.254:35595)
0809 15:35:59.583696 (+ 661us) server_negotiation.cc:206] Beginning negotiation
0809 15:35:59.583712 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0809 15:35:59.583748 (+ 36us) server_negotiation.cc:407] Connection header received
0809 15:35:59.584493 (+ 745us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 15:35:59.584512 (+ 19us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 15:35:59.584910 (+ 398us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 15:35:59.585188 (+ 278us) server_negotiation.cc:226] Negotiated authn=SASL
0809 15:35:59.585641 (+ 453us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:35:59.586632 (+ 991us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:35:59.587696 (+ 1064us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:35:59.588172 (+ 476us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:35:59.588359 (+ 187us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:35:59.589261 (+ 902us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 15:35:59.589266 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 15:35:59.589278 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 15:35:59.589720 (+ 442us) server_negotiation.cc:922] calling sasl_server_start()
0809 15:35:59.589780 (+ 60us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 15:35:59.589799 (+ 19us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 15:35:59.589921 (+ 122us) server_negotiation.cc:1035] Waiting for connection context
0809 15:35:59.590065 (+ 144us) server_negotiation.cc:299] Negotiation successful
0809 15:35:59.590367 (+ 302us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":302,"thread_start_us":141,"threads_started":1}
I20260809 15:35:59.593071 9236 raft_consensus.cc:348] T 00000000000000000000000000000000 P d0f8335ba3964bd4bf64ded80c67274d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d0f8335ba3964bd4bf64ded80c67274d" member_type: VOTER last_known_addr { host: "127.3.89.254" port: 35595 } }
I20260809 15:35:59.593313 9236 raft_consensus.cc:374] T 00000000000000000000000000000000 P d0f8335ba3964bd4bf64ded80c67274d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 15:35:59.593395 9236 raft_consensus.cc:729] T 00000000000000000000000000000000 P d0f8335ba3964bd4bf64ded80c67274d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d0f8335ba3964bd4bf64ded80c67274d, State: Initialized, Role: FOLLOWER
I20260809 15:35:59.593902 9236 consensus_queue.cc:260] T 00000000000000000000000000000000 P d0f8335ba3964bd4bf64ded80c67274d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d0f8335ba3964bd4bf64ded80c67274d" member_type: VOTER last_known_addr { host: "127.3.89.254" port: 35595 } }
I20260809 15:35:59.594061 9236 raft_consensus.cc:388] T 00000000000000000000000000000000 P d0f8335ba3964bd4bf64ded80c67274d [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 15:35:59.594131 9236 raft_consensus.cc:482] T 00000000000000000000000000000000 P d0f8335ba3964bd4bf64ded80c67274d [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 15:35:59.594229 9236 raft_consensus.cc:3037] T 00000000000000000000000000000000 P d0f8335ba3964bd4bf64ded80c67274d [term 0 FOLLOWER]: Advancing to term 1
I20260809 15:35:59.594822 3431 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260809 15:35:59.676445 9236 raft_consensus.cc:504] T 00000000000000000000000000000000 P d0f8335ba3964bd4bf64ded80c67274d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d0f8335ba3964bd4bf64ded80c67274d" member_type: VOTER last_known_addr { host: "127.3.89.254" port: 35595 } }
I20260809 15:35:59.676978 9236 leader_election.cc:302] T 00000000000000000000000000000000 P d0f8335ba3964bd4bf64ded80c67274d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: d0f8335ba3964bd4bf64ded80c67274d; no voters:
I20260809 15:35:59.677373 9236 leader_election.cc:288] T 00000000000000000000000000000000 P d0f8335ba3964bd4bf64ded80c67274d [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 15:35:59.677538 9241 raft_consensus.cc:2781] T 00000000000000000000000000000000 P d0f8335ba3964bd4bf64ded80c67274d [term 1 FOLLOWER]: Leader election won for term 1
I20260809 15:35:59.677790 9241 raft_consensus.cc:686] T 00000000000000000000000000000000 P d0f8335ba3964bd4bf64ded80c67274d [term 1 LEADER]: Becoming Leader. State: Replica: d0f8335ba3964bd4bf64ded80c67274d, State: Running, Role: LEADER
I20260809 15:35:59.678238 9241 consensus_queue.cc:237] T 00000000000000000000000000000000 P d0f8335ba3964bd4bf64ded80c67274d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d0f8335ba3964bd4bf64ded80c67274d" member_type: VOTER last_known_addr { host: "127.3.89.254" port: 35595 } }
I20260809 15:35:59.678877 9236 sys_catalog.cc:564] T 00000000000000000000000000000000 P d0f8335ba3964bd4bf64ded80c67274d [sys.catalog]: configured and running, proceeding with master startup.
I20260809 15:35:59.680143 9242 sys_catalog.cc:455] T 00000000000000000000000000000000 P d0f8335ba3964bd4bf64ded80c67274d [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "d0f8335ba3964bd4bf64ded80c67274d" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d0f8335ba3964bd4bf64ded80c67274d" member_type: VOTER last_known_addr { host: "127.3.89.254" port: 35595 } } }
I20260809 15:35:59.680294 9242 sys_catalog.cc:458] T 00000000000000000000000000000000 P d0f8335ba3964bd4bf64ded80c67274d [sys.catalog]: This master's current role is: LEADER
I20260809 15:35:59.680696 9243 sys_catalog.cc:455] T 00000000000000000000000000000000 P d0f8335ba3964bd4bf64ded80c67274d [sys.catalog]: SysCatalogTable state changed. Reason: New leader d0f8335ba3964bd4bf64ded80c67274d. Latest consensus state: current_term: 1 leader_uuid: "d0f8335ba3964bd4bf64ded80c67274d" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d0f8335ba3964bd4bf64ded80c67274d" member_type: VOTER last_known_addr { host: "127.3.89.254" port: 35595 } } }
I20260809 15:35:59.680873 9243 sys_catalog.cc:458] T 00000000000000000000000000000000 P d0f8335ba3964bd4bf64ded80c67274d [sys.catalog]: This master's current role is: LEADER
I20260809 15:35:59.680857 9251 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 15:35:59.684362 9251 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 15:35:59.690281 9251 catalog_manager.cc:1347] Generated new cluster ID: 1347ed98542f4848ad2e6957a2c17a4c
I20260809 15:35:59.690362 9251 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 15:35:59.702868 9251 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 15:35:59.704142 9251 catalog_manager.cc:1504] Loading token signing keys...
I20260809 15:35:59.717381 9251 catalog_manager.cc:5868] T 00000000000000000000000000000000 P d0f8335ba3964bd4bf64ded80c67274d: Generated new TSK 0
I20260809 15:35:59.718312 9251 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260809 15:35:59.744943 9239 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:35:59.739833 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38212 (local address 127.3.89.254:35595)
0809 15:35:59.739990 (+ 157us) server_negotiation.cc:206] Beginning negotiation
0809 15:35:59.739995 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 15:35:59.740151 (+ 156us) server_negotiation.cc:407] Connection header received
0809 15:35:59.740326 (+ 175us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 15:35:59.740329 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 15:35:59.740376 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 15:35:59.740468 (+ 92us) server_negotiation.cc:226] Negotiated authn=JWT
0809 15:35:59.740725 (+ 257us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:35:59.741257 (+ 532us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:35:59.742498 (+ 1241us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:35:59.742717 (+ 219us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:35:59.742794 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:35:59.743173 (+ 379us) server_negotiation.cc:365] Received JWT_EXCHANGE NegotiatePB request
0809 15:35:59.744573 (+ 1400us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:35853/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
0809 15:35:59.744804 (+ 231us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:38212: Failed to load JWKS: Error downloading JWKS from 'https://localhost:35853/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
Metrics: {"server-negotiator.queue_time_us":61}
W20260809 15:35:59.745019 9239 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:38212: Failed to load JWKS: Error downloading JWKS from 'https://localhost:35853/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
W20260809 15:35:59.746738 9267 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 15:35:59.739747 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.89.254:35595 (local address 127.0.0.1:38212)
0809 15:35:59.740070 (+ 323us) negotiation.cc:107] Waiting for socket to connect
0809 15:35:59.740085 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0809 15:35:59.740204 (+ 119us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 15:35:59.740490 (+ 286us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 15:35:59.740493 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 15:35:59.740502 (+ 9us) client_negotiation.cc:189] Negotiated authn=JWT
0809 15:35:59.740627 (+ 125us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:35:59.740634 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:35:59.741369 (+ 735us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:35:59.741372 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:35:59.742377 (+ 1005us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:35:59.742388 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:35:59.742863 (+ 475us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:35:59.742867 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:35:59.742984 (+ 117us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:35:59.743015 (+ 31us) client_negotiation.cc:252] Sending JWT_EXCHANGE NegotiatePB request
0809 15:35:59.744751 (+ 1736us) client_negotiation.cc:283] Received error response from server: Runtime error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:35853/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
0809 15:35:59.746554 (+ 1803us) negotiation.cc:327] Negotiation complete: Runtime error: Client connection negotiation failed: client connection to 127.3.89.254:35595: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:35853/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
Metrics: {"client-negotiator.queue_time_us":226,"mutex_wait_us":1583,"thread_start_us":135,"threads_started":1}
I20260809 15:35:59.747370 3431 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskm314YR/build/debug/bin/kudu with pid 9164
E20260809 15:35:59.758494 9268 webserver.cc:533] Webserver: sslize failed:
E20260809 15:35:59.758577 9268 webserver.cc:533] Webserver: error shutting down SSL connection: error:140E0197:SSL routines:SSL_shutdown:shutdown while in init
2026-08-09T15:35:59Z chronyd exiting
[ OK ] SecurityITest.TestJwtMiniClusterWithUntrustedCert (396 ms)
[ RUN ] SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS
2026-08-09T15:35:59Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T15:35:59Z Disabled control of system clock
I20260809 15:35:59.800959 3431 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskm314YR/build/debug/bin/kudu
/tmp/dist-test-taskm314YR/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786289732823074-3431-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786289732823074-3431-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786289732823074-3431-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786289732823074-3431-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.3.89.254:36333
--webserver_interface=127.3.89.254
--webserver_port=0
--builtin_ntp_servers=127.3.89.212:42209
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.3.89.254:36333
--enable_jwt_token_auth=true
--jwks_url=default.url
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--trusted_certificate_file=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786289732823074-3431-0/testchainca.cert with env {}
W20260809 15:35:59.918730 9272 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 15:35:59.919063 9272 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 15:35:59.919109 9272 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 15:35:59.923331 9272 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 15:35:59.923393 9272 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 15:35:59.923408 9272 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 15:35:59.923424 9272 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 15:35:59.923460 9272 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
W20260809 15:35:59.923528 9272 flags.cc:425] Enabled experimental flag: --enable_jwt_token_auth=true
W20260809 15:35:59.923593 9272 flags.cc:425] Enabled experimental flag: --jwks_url=default.url
I20260809 15:35:59.929733 9272 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.3.89.212:42209
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786289732823074-3431-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786289732823074-3431-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.3.89.254:36333
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.3.89.254:36333
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--enable_jwt_token_auth=true
--jwks_url=default.url
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786289732823074-3431-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.3.89.254
--webserver_port=0
--trusted_certificate_file=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786289732823074-3431-0/testchainca.cert
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.9272
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786289732823074-3431-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 09 Aug 2026 14:46:55 UTC on ccycloud.quasar-yswkmf.root.comops.site
build id 4576540
I20260809 15:35:59.930811 9272 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 15:35:59.931902 9272 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 15:35:59.940860 9278 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 15:35:59.940877 9280 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 15:35:59.940923 9277 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 15:35:59.940927 9272 server_base.cc:1034] running on GCE node
I20260809 15:35:59.941725 9272 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 15:35:59.942934 9272 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 15:35:59.944176 9272 hybrid_clock.cc:648] HybridClock initialized: now 1786289759944148 us; error 40 us; skew 500 ppm
I20260809 15:35:59.946141 9272 webserver.cc:460] Webserver started at http://127.3.89.254:38361/ using document root <none> and password file <none>
I20260809 15:35:59.946764 9272 fs_manager.cc:359] Metadata directory not provided
I20260809 15:35:59.946857 9272 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 15:35:59.947173 9272 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 15:35:59.949265 9272 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786289732823074-3431-0/minicluster-data/master-0/data/instance:
uuid: "3b5a5ebab53641c9b091d7d214463893"
format_stamp: "Formatted at 2026-08-09 15:35:59 on dist-test-slave-2w3w"
server_key: "cd1750909a1ec29687a72b596e313666"
server_key_iv: "047d3870d745617322e445e3c508293c"
server_key_version: "encryptionkey@0"
I20260809 15:35:59.949868 9272 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786289732823074-3431-0/minicluster-data/master-0/wal/instance:
uuid: "3b5a5ebab53641c9b091d7d214463893"
format_stamp: "Formatted at 2026-08-09 15:35:59 on dist-test-slave-2w3w"
server_key: "cd1750909a1ec29687a72b596e313666"
server_key_iv: "047d3870d745617322e445e3c508293c"
server_key_version: "encryptionkey@0"
I20260809 15:35:59.953613 9272 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.002s sys 0.004s
I20260809 15:35:59.956091 9287 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 15:35:59.957315 9272 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260809 15:35:59.957458 9272 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786289732823074-3431-0/minicluster-data/master-0/data,/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786289732823074-3431-0/minicluster-data/master-0/wal
uuid: "3b5a5ebab53641c9b091d7d214463893"
format_stamp: "Formatted at 2026-08-09 15:35:59 on dist-test-slave-2w3w"
server_key: "cd1750909a1ec29687a72b596e313666"
server_key_iv: "047d3870d745617322e445e3c508293c"
server_key_version: "encryptionkey@0"
I20260809 15:35:59.957578 9272 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786289732823074-3431-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786289732823074-3431-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786289732823074-3431-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 15:35:59.990213 9272 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 15:35:59.991000 9272 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 15:35:59.991210 9272 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 15:35:59.998241 9272 rpc_server.cc:307] RPC server started. Bound to: 127.3.89.254:36333
I20260809 15:35:59.998260 9346 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.3.89.254:36333 every 8 connection(s)
I20260809 15:35:59.999225 9272 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786289732823074-3431-0/minicluster-data/master-0/data/info.pb
I20260809 15:36:00.002415 9347 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 15:36:00.007565 3431 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskm314YR/build/debug/bin/kudu as pid 9272
I20260809 15:36:00.007673 3431 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786289732823074-3431-0/minicluster-data/master-0/wal/instance
I20260809 15:36:00.007984 3431 external_mini_cluster.cc:1428] Setting key e73d7abab034e8bcad8d0173441b1c4c
I20260809 15:36:00.008265 9347 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 3b5a5ebab53641c9b091d7d214463893: Bootstrap starting.
I20260809 15:36:00.010713 9347 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 3b5a5ebab53641c9b091d7d214463893: Neither blocks nor log segments found. Creating new log.
I20260809 15:36:00.011585 9347 log.cc:824] T 00000000000000000000000000000000 P 3b5a5ebab53641c9b091d7d214463893: Log is configured to *not* fsync() on all Append() calls
I20260809 15:36:00.013656 9347 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 3b5a5ebab53641c9b091d7d214463893: No bootstrap required, opened a new log
I20260809 15:36:00.017498 9350 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:36:00.009454 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40460 (local address 127.3.89.254:36333)
0809 15:36:00.010123 (+ 669us) server_negotiation.cc:206] Beginning negotiation
0809 15:36:00.010137 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0809 15:36:00.010173 (+ 36us) server_negotiation.cc:407] Connection header received
0809 15:36:00.010863 (+ 690us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 15:36:00.010882 (+ 19us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 15:36:00.011326 (+ 444us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 15:36:00.011599 (+ 273us) server_negotiation.cc:226] Negotiated authn=SASL
0809 15:36:00.012002 (+ 403us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:36:00.012971 (+ 969us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:36:00.014168 (+ 1197us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:36:00.014641 (+ 473us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:36:00.014802 (+ 161us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:36:00.015719 (+ 917us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 15:36:00.015724 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 15:36:00.015736 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 15:36:00.016143 (+ 407us) server_negotiation.cc:922] calling sasl_server_start()
0809 15:36:00.016197 (+ 54us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 15:36:00.016217 (+ 20us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 15:36:00.016297 (+ 80us) server_negotiation.cc:1035] Waiting for connection context
0809 15:36:00.016463 (+ 166us) server_negotiation.cc:299] Negotiation successful
0809 15:36:00.016757 (+ 294us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":309,"thread_start_us":152,"threads_started":1}
I20260809 15:36:00.017850 9347 raft_consensus.cc:348] T 00000000000000000000000000000000 P 3b5a5ebab53641c9b091d7d214463893 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3b5a5ebab53641c9b091d7d214463893" member_type: VOTER last_known_addr { host: "127.3.89.254" port: 36333 } }
I20260809 15:36:00.018091 9347 raft_consensus.cc:374] T 00000000000000000000000000000000 P 3b5a5ebab53641c9b091d7d214463893 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 15:36:00.018180 9347 raft_consensus.cc:729] T 00000000000000000000000000000000 P 3b5a5ebab53641c9b091d7d214463893 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3b5a5ebab53641c9b091d7d214463893, State: Initialized, Role: FOLLOWER
I20260809 15:36:00.018648 9347 consensus_queue.cc:260] T 00000000000000000000000000000000 P 3b5a5ebab53641c9b091d7d214463893 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3b5a5ebab53641c9b091d7d214463893" member_type: VOTER last_known_addr { host: "127.3.89.254" port: 36333 } }
I20260809 15:36:00.018807 9347 raft_consensus.cc:388] T 00000000000000000000000000000000 P 3b5a5ebab53641c9b091d7d214463893 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 15:36:00.018874 9347 raft_consensus.cc:482] T 00000000000000000000000000000000 P 3b5a5ebab53641c9b091d7d214463893 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 15:36:00.018990 9347 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 3b5a5ebab53641c9b091d7d214463893 [term 0 FOLLOWER]: Advancing to term 1
I20260809 15:36:00.020148 9347 raft_consensus.cc:504] T 00000000000000000000000000000000 P 3b5a5ebab53641c9b091d7d214463893 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3b5a5ebab53641c9b091d7d214463893" member_type: VOTER last_known_addr { host: "127.3.89.254" port: 36333 } }
I20260809 15:36:00.020524 9347 leader_election.cc:302] T 00000000000000000000000000000000 P 3b5a5ebab53641c9b091d7d214463893 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 3b5a5ebab53641c9b091d7d214463893; no voters:
I20260809 15:36:00.020593 3431 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260809 15:36:00.020852 9347 leader_election.cc:288] T 00000000000000000000000000000000 P 3b5a5ebab53641c9b091d7d214463893 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 15:36:00.020947 9352 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 3b5a5ebab53641c9b091d7d214463893 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 15:36:00.021162 9352 raft_consensus.cc:686] T 00000000000000000000000000000000 P 3b5a5ebab53641c9b091d7d214463893 [term 1 LEADER]: Becoming Leader. State: Replica: 3b5a5ebab53641c9b091d7d214463893, State: Running, Role: LEADER
I20260809 15:36:00.021544 9352 consensus_queue.cc:237] T 00000000000000000000000000000000 P 3b5a5ebab53641c9b091d7d214463893 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3b5a5ebab53641c9b091d7d214463893" member_type: VOTER last_known_addr { host: "127.3.89.254" port: 36333 } }
I20260809 15:36:00.022113 9347 sys_catalog.cc:564] T 00000000000000000000000000000000 P 3b5a5ebab53641c9b091d7d214463893 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 15:36:00.022764 9352 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3b5a5ebab53641c9b091d7d214463893 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 3b5a5ebab53641c9b091d7d214463893. Latest consensus state: current_term: 1 leader_uuid: "3b5a5ebab53641c9b091d7d214463893" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3b5a5ebab53641c9b091d7d214463893" member_type: VOTER last_known_addr { host: "127.3.89.254" port: 36333 } } }
I20260809 15:36:00.022960 9352 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3b5a5ebab53641c9b091d7d214463893 [sys.catalog]: This master's current role is: LEADER
I20260809 15:36:00.023239 9353 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3b5a5ebab53641c9b091d7d214463893 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "3b5a5ebab53641c9b091d7d214463893" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3b5a5ebab53641c9b091d7d214463893" member_type: VOTER last_known_addr { host: "127.3.89.254" port: 36333 } } }
I20260809 15:36:00.023408 9353 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3b5a5ebab53641c9b091d7d214463893 [sys.catalog]: This master's current role is: LEADER
I20260809 15:36:00.023523 9361 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 15:36:00.026613 9361 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 15:36:00.032334 9361 catalog_manager.cc:1347] Generated new cluster ID: 721b961b13824747b3fb96abb871296c
I20260809 15:36:00.032436 9361 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 15:36:00.080328 9361 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 15:36:00.081344 9361 catalog_manager.cc:1504] Loading token signing keys...
I20260809 15:36:00.093094 9361 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 3b5a5ebab53641c9b091d7d214463893: Generated new TSK 0
I20260809 15:36:00.093842 9361 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260809 15:36:00.964161 9350 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:36:00.101681 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40468 (local address 127.3.89.254:36333)
0809 15:36:00.101822 (+ 141us) server_negotiation.cc:206] Beginning negotiation
0809 15:36:00.101827 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 15:36:00.102024 (+ 197us) server_negotiation.cc:407] Connection header received
0809 15:36:00.102194 (+ 170us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 15:36:00.102196 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 15:36:00.102241 (+ 45us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 15:36:00.102325 (+ 84us) server_negotiation.cc:226] Negotiated authn=JWT
0809 15:36:00.102587 (+ 262us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:36:00.103159 (+ 572us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:36:00.104393 (+ 1234us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:36:00.104620 (+ 227us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:36:00.104699 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:36:00.105033 (+ 334us) server_negotiation.cc:365] Received JWT_EXCHANGE NegotiatePB request
0809 15:36:00.963746 (+858713us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
0809 15:36:00.964033 (+ 287us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:40468: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
Metrics: {"server-negotiator.queue_time_us":58}
W20260809 15:36:00.964278 9350 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:40468: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
W20260809 15:36:00.964711 9377 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 15:36:00.101611 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.89.254:36333 (local address 127.0.0.1:40468)
0809 15:36:00.101928 (+ 317us) negotiation.cc:107] Waiting for socket to connect
0809 15:36:00.101944 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0809 15:36:00.102057 (+ 113us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 15:36:00.102358 (+ 301us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 15:36:00.102361 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 15:36:00.102370 (+ 9us) client_negotiation.cc:189] Negotiated authn=JWT
0809 15:36:00.102483 (+ 113us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:36:00.102490 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:36:00.103303 (+ 813us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:36:00.103306 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:36:00.104273 (+ 967us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:36:00.104293 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:36:00.104744 (+ 451us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:36:00.104748 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:36:00.104844 (+ 96us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:36:00.104872 (+ 28us) client_negotiation.cc:252] Sending JWT_EXCHANGE NegotiatePB request
0809 15:36:00.963993 (+859121us) client_negotiation.cc:283] Received error response from server: Runtime error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
0809 15:36:00.964202 (+ 209us) negotiation.cc:327] Negotiation complete: Runtime error: Client connection negotiation failed: client connection to 127.3.89.254:36333: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
Metrics: {"client-negotiator.queue_time_us":224,"thread_start_us":123,"threads_started":1}
I20260809 15:36:00.965698 3431 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskm314YR/build/debug/bin/kudu with pid 9272
2026-08-09T15:36:00Z chronyd exiting
[ OK ] SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS (1197 ms)
[ RUN ] SecurityITest.TestWorldReadableKeytab
[ OK ] SecurityITest.TestWorldReadableKeytab (127 ms)
[ RUN ] SecurityITest.TestWorldReadablePrivateKey
[ OK ] SecurityITest.TestWorldReadablePrivateKey (124 ms)
[ RUN ] SecurityITest.TestCorruptKerberosCC
Loading random data
Initializing database '/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786289732823074-3431-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 15:36:01 dist-test-slave-2w3w krb5kdc[9389](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 15:36:01 dist-test-slave-2w3w krb5kdc[9389](info): set up 2 sockets
Aug 09 15:36:01 dist-test-slave-2w3w krb5kdc[9389](info): commencing operation
krb5kdc: starting...
W20260809 15:36:02.296798 3431 mini_kdc.cc:121] Time spent starting KDC: real 1.057s user 0.003s sys 0.004s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786289732823074-3431-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786289732823074-3431-0/krb5kdc/test-user.keytab.
Aug 09 15:36:02 dist-test-slave-2w3w krb5kdc[9389](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786289762, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T15:36:02Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T15:36:02Z Disabled control of system clock
WARNING: no policy specified for kudu/127.3.89.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.3.89.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.3.89.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786289732823074-3431-0/krb5kdc/kudu_127.3.89.254.keytab.
Entry for principal kudu/127.3.89.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786289732823074-3431-0/krb5kdc/kudu_127.3.89.254.keytab.
I20260809 15:36:02.426484 3431 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskm314YR/build/debug/bin/kudu
/tmp/dist-test-taskm314YR/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786289732823074-3431-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786289732823074-3431-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786289732823074-3431-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786289732823074-3431-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.3.89.254:45161
--webserver_interface=127.3.89.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786289732823074-3431-0/krb5kdc/kudu_127.3.89.254.keytab
--principal=kudu/127.3.89.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.3.89.212:42519
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.3.89.254:45161
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786289732823074-3431-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786289732823074-3431-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786289732823074-3431-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 15:36:02.552017 9403 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 15:36:02.552278 9403 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 15:36:02.552320 9403 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 15:36:02.556314 9403 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 15:36:02.556370 9403 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 15:36:02.556391 9403 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 15:36:02.556407 9403 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 15:36:02.556422 9403 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 15:36:02.561650 9403 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.3.89.212:42519
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786289732823074-3431-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786289732823074-3431-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.3.89.254:45161
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786289732823074-3431-0/krb5kdc/kudu_127.3.89.254.keytab
--principal=kudu/127.3.89.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.3.89.254:45161
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786289732823074-3431-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.3.89.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.9403
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786289732823074-3431-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 09 Aug 2026 14:46:55 UTC on ccycloud.quasar-yswkmf.root.comops.site
build id 4576540
I20260809 15:36:02.562660 9403 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 15:36:02.563686 9403 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 15:36:02.572054 9410 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 15:36:02.572064 9412 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 15:36:02.572073 9409 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 15:36:02.572868 9403 server_base.cc:1034] running on GCE node
I20260809 15:36:02.573628 9403 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 15:36:02.574842 9403 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 15:36:02.576031 9403 hybrid_clock.cc:648] HybridClock initialized: now 1786289762576013 us; error 30 us; skew 500 ppm
Aug 09 15:36:02 dist-test-slave-2w3w krb5kdc[9389](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786289762, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.89.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 15:36:02.578863 9403 init.cc:377] Logged in from keytab as kudu/127.3.89.254@KRBTEST.COM (short username kudu)
I20260809 15:36:02.579723 9403 webserver.cc:460] Webserver started at http://127.3.89.254:37457/ using document root <none> and password file <none>
I20260809 15:36:02.580281 9403 fs_manager.cc:359] Metadata directory not provided
I20260809 15:36:02.580374 9403 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 15:36:02.580605 9403 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 15:36:02.582665 9403 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786289732823074-3431-0/minicluster-data/master-0/data/instance:
uuid: "8e2e3a838a6e41d28d6c459417ba723a"
format_stamp: "Formatted at 2026-08-09 15:36:02 on dist-test-slave-2w3w"
server_key: "a15ab9f3b9a7db45225c8f9681c00a35"
server_key_iv: "8d8439e7f7fe0dabc2712c84d3f7f8b0"
server_key_version: "encryptionkey@0"
I20260809 15:36:02.583292 9403 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786289732823074-3431-0/minicluster-data/master-0/wal/instance:
uuid: "8e2e3a838a6e41d28d6c459417ba723a"
format_stamp: "Formatted at 2026-08-09 15:36:02 on dist-test-slave-2w3w"
server_key: "a15ab9f3b9a7db45225c8f9681c00a35"
server_key_iv: "8d8439e7f7fe0dabc2712c84d3f7f8b0"
server_key_version: "encryptionkey@0"
I20260809 15:36:02.586937 9403 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.002s sys 0.001s
I20260809 15:36:02.589504 9420 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 15:36:02.591071 9403 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260809 15:36:02.591208 9403 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786289732823074-3431-0/minicluster-data/master-0/data,/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786289732823074-3431-0/minicluster-data/master-0/wal
uuid: "8e2e3a838a6e41d28d6c459417ba723a"
format_stamp: "Formatted at 2026-08-09 15:36:02 on dist-test-slave-2w3w"
server_key: "a15ab9f3b9a7db45225c8f9681c00a35"
server_key_iv: "8d8439e7f7fe0dabc2712c84d3f7f8b0"
server_key_version: "encryptionkey@0"
I20260809 15:36:02.591326 9403 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786289732823074-3431-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786289732823074-3431-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786289732823074-3431-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 15:36:02.601495 9403 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 15:36:02.604729 9403 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 15:36:02.604959 9403 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 15:36:02.611825 9403 rpc_server.cc:307] RPC server started. Bound to: 127.3.89.254:45161
I20260809 15:36:02.611871 9477 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.3.89.254:45161 every 8 connection(s)
I20260809 15:36:02.612728 9403 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786289732823074-3431-0/minicluster-data/master-0/data/info.pb
I20260809 15:36:02.613615 3431 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskm314YR/build/debug/bin/kudu as pid 9403
I20260809 15:36:02.613749 3431 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786289732823074-3431-0/minicluster-data/master-0/wal/instance
I20260809 15:36:02.614028 3431 external_mini_cluster.cc:1428] Setting key 8b7093d9938df16f0876a5bcabea201f
I20260809 15:36:02.615787 9478 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 15:36:02.622298 9478 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8e2e3a838a6e41d28d6c459417ba723a: Bootstrap starting.
Aug 09 15:36:02 dist-test-slave-2w3w krb5kdc[9389](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786289762, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.3.89.254@KRBTEST.COM
I20260809 15:36:02.624939 9478 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 8e2e3a838a6e41d28d6c459417ba723a: Neither blocks nor log segments found. Creating new log.
I20260809 15:36:02.625850 9478 log.cc:824] T 00000000000000000000000000000000 P 8e2e3a838a6e41d28d6c459417ba723a: Log is configured to *not* fsync() on all Append() calls
I20260809 15:36:02.628244 9478 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8e2e3a838a6e41d28d6c459417ba723a: No bootstrap required, opened a new log
I20260809 15:36:02.629410 9482 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:36:02.615451 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48578 (local address 127.3.89.254:45161)
0809 15:36:02.615892 (+ 441us) server_negotiation.cc:206] Beginning negotiation
0809 15:36:02.615911 (+ 19us) server_negotiation.cc:399] Waiting for connection header
0809 15:36:02.615952 (+ 41us) server_negotiation.cc:407] Connection header received
0809 15:36:02.616771 (+ 819us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 15:36:02.616792 (+ 21us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 15:36:02.617196 (+ 404us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 15:36:02.617540 (+ 344us) server_negotiation.cc:226] Negotiated authn=SASL
0809 15:36:02.618208 (+ 668us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:36:02.619686 (+ 1478us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:36:02.620560 (+ 874us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:36:02.621269 (+ 709us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:36:02.621422 (+ 153us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:36:02.624194 (+ 2772us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 15:36:02.624200 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 15:36:02.624213 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 15:36:02.624244 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0809 15:36:02.626239 (+ 1995us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 15:36:02.626769 (+ 530us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 15:36:02.626775 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 15:36:02.626780 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0809 15:36:02.627016 (+ 236us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 15:36:02.627288 (+ 272us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 15:36:02.627291 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 15:36:02.627293 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 15:36:02.627772 (+ 479us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 15:36:02.627966 (+ 194us) server_negotiation.cc:1035] Waiting for connection context
0809 15:36:02.628207 (+ 241us) server_negotiation.cc:299] Negotiation successful
0809 15:36:02.628503 (+ 296us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":280,"thread_start_us":124,"threads_started":1}
I20260809 15:36:02.632032 9478 raft_consensus.cc:348] T 00000000000000000000000000000000 P 8e2e3a838a6e41d28d6c459417ba723a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8e2e3a838a6e41d28d6c459417ba723a" member_type: VOTER last_known_addr { host: "127.3.89.254" port: 45161 } }
I20260809 15:36:02.632269 9478 raft_consensus.cc:374] T 00000000000000000000000000000000 P 8e2e3a838a6e41d28d6c459417ba723a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 15:36:02.632308 9478 raft_consensus.cc:729] T 00000000000000000000000000000000 P 8e2e3a838a6e41d28d6c459417ba723a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8e2e3a838a6e41d28d6c459417ba723a, State: Initialized, Role: FOLLOWER
I20260809 15:36:02.632699 9478 consensus_queue.cc:260] T 00000000000000000000000000000000 P 8e2e3a838a6e41d28d6c459417ba723a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8e2e3a838a6e41d28d6c459417ba723a" member_type: VOTER last_known_addr { host: "127.3.89.254" port: 45161 } }
I20260809 15:36:02.632819 9478 raft_consensus.cc:388] T 00000000000000000000000000000000 P 8e2e3a838a6e41d28d6c459417ba723a [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 15:36:02.632869 9478 raft_consensus.cc:482] T 00000000000000000000000000000000 P 8e2e3a838a6e41d28d6c459417ba723a [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 15:36:02.632931 9478 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 8e2e3a838a6e41d28d6c459417ba723a [term 0 FOLLOWER]: Advancing to term 1
I20260809 15:36:02.633742 9478 raft_consensus.cc:504] T 00000000000000000000000000000000 P 8e2e3a838a6e41d28d6c459417ba723a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8e2e3a838a6e41d28d6c459417ba723a" member_type: VOTER last_known_addr { host: "127.3.89.254" port: 45161 } }
I20260809 15:36:02.634019 9478 leader_election.cc:302] T 00000000000000000000000000000000 P 8e2e3a838a6e41d28d6c459417ba723a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 8e2e3a838a6e41d28d6c459417ba723a; no voters:
I20260809 15:36:02.634270 9478 leader_election.cc:288] T 00000000000000000000000000000000 P 8e2e3a838a6e41d28d6c459417ba723a [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 15:36:02.634411 9484 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 8e2e3a838a6e41d28d6c459417ba723a [term 1 FOLLOWER]: Leader election won for term 1
I20260809 15:36:02.634660 9484 raft_consensus.cc:686] T 00000000000000000000000000000000 P 8e2e3a838a6e41d28d6c459417ba723a [term 1 LEADER]: Becoming Leader. State: Replica: 8e2e3a838a6e41d28d6c459417ba723a, State: Running, Role: LEADER
I20260809 15:36:02.635070 9484 consensus_queue.cc:237] T 00000000000000000000000000000000 P 8e2e3a838a6e41d28d6c459417ba723a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8e2e3a838a6e41d28d6c459417ba723a" member_type: VOTER last_known_addr { host: "127.3.89.254" port: 45161 } }
I20260809 15:36:02.635532 9478 sys_catalog.cc:564] T 00000000000000000000000000000000 P 8e2e3a838a6e41d28d6c459417ba723a [sys.catalog]: configured and running, proceeding with master startup.
I20260809 15:36:02.636855 9486 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8e2e3a838a6e41d28d6c459417ba723a [sys.catalog]: SysCatalogTable state changed. Reason: New leader 8e2e3a838a6e41d28d6c459417ba723a. Latest consensus state: current_term: 1 leader_uuid: "8e2e3a838a6e41d28d6c459417ba723a" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8e2e3a838a6e41d28d6c459417ba723a" member_type: VOTER last_known_addr { host: "127.3.89.254" port: 45161 } } }
I20260809 15:36:02.636994 9486 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8e2e3a838a6e41d28d6c459417ba723a [sys.catalog]: This master's current role is: LEADER
I20260809 15:36:02.637281 9485 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8e2e3a838a6e41d28d6c459417ba723a [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "8e2e3a838a6e41d28d6c459417ba723a" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8e2e3a838a6e41d28d6c459417ba723a" member_type: VOTER last_known_addr { host: "127.3.89.254" port: 45161 } } }
I20260809 15:36:02.637377 9485 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8e2e3a838a6e41d28d6c459417ba723a [sys.catalog]: This master's current role is: LEADER
I20260809 15:36:02.637369 9493 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 15:36:02.640538 9493 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 15:36:02.645763 9493 catalog_manager.cc:1347] Generated new cluster ID: d3db9bf3a8a640b381ad833bc4e11a06
I20260809 15:36:02.645828 9493 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 15:36:02.677217 9493 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 15:36:02.678510 9493 catalog_manager.cc:1504] Loading token signing keys...
I20260809 15:36:02.690105 9493 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 8e2e3a838a6e41d28d6c459417ba723a: Generated new TSK 0
I20260809 15:36:02.690841 9493 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.3.89.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.3.89.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.3.89.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786289732823074-3431-0/krb5kdc/kudu_127.3.89.193.keytab.
Entry for principal kudu/127.3.89.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786289732823074-3431-0/krb5kdc/kudu_127.3.89.193.keytab.
I20260809 15:36:02.727739 3431 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskm314YR/build/debug/bin/kudu
/tmp/dist-test-taskm314YR/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786289732823074-3431-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786289732823074-3431-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786289732823074-3431-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786289732823074-3431-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.3.89.193:0
--local_ip_for_outbound_sockets=127.3.89.193
--webserver_interface=127.3.89.193
--webserver_port=0
--tserver_master_addrs=127.3.89.254:45161
--keytab_file=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786289732823074-3431-0/krb5kdc/kudu_127.3.89.193.keytab
--principal=kudu/127.3.89.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.3.89.212:42519
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786289732823074-3431-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786289732823074-3431-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786289732823074-3431-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 15:36:02.862195 9506 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 15:36:02.862455 9506 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 15:36:02.862496 9506 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 15:36:02.866459 9506 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 15:36:02.866511 9506 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 15:36:02.866621 9506 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.3.89.193
I20260809 15:36:02.871699 9506 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.3.89.212:42519
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786289732823074-3431-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786289732823074-3431-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786289732823074-3431-0/krb5kdc/kudu_127.3.89.193.keytab
--principal=kudu/127.3.89.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.3.89.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786289732823074-3431-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.3.89.193
--webserver_port=0
--tserver_master_addrs=127.3.89.254:45161
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.9506
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.3.89.193
--log_dir=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786289732823074-3431-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 09 Aug 2026 14:46:55 UTC on ccycloud.quasar-yswkmf.root.comops.site
build id 4576540
I20260809 15:36:02.872686 9506 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 15:36:02.873664 9506 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 15:36:02.882989 9511 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 15:36:02.883014 9514 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 15:36:02.883014 9512 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 15:36:02.883249 9506 server_base.cc:1034] running on GCE node
I20260809 15:36:02.883668 9506 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 15:36:02.884208 9506 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 15:36:02.885408 9506 hybrid_clock.cc:648] HybridClock initialized: now 1786289762885386 us; error 32 us; skew 500 ppm
Aug 09 15:36:02 dist-test-slave-2w3w krb5kdc[9389](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786289762, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.89.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 15:36:02.888317 9506 init.cc:377] Logged in from keytab as kudu/127.3.89.193@KRBTEST.COM (short username kudu)
I20260809 15:36:02.889138 9506 webserver.cc:460] Webserver started at http://127.3.89.193:41749/ using document root <none> and password file <none>
I20260809 15:36:02.889636 9506 fs_manager.cc:359] Metadata directory not provided
I20260809 15:36:02.889693 9506 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 15:36:02.889859 9506 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 15:36:02.891901 9506 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786289732823074-3431-0/minicluster-data/ts-0/data/instance:
uuid: "4b3563c4e6e749448e98a4042dc9dace"
format_stamp: "Formatted at 2026-08-09 15:36:02 on dist-test-slave-2w3w"
server_key: "d15f4048bac07701997489e9248f07b9"
server_key_iv: "9fb25b4916ec49bbbafd466cbbf7f31d"
server_key_version: "encryptionkey@0"
I20260809 15:36:02.892407 9506 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786289732823074-3431-0/minicluster-data/ts-0/wal/instance:
uuid: "4b3563c4e6e749448e98a4042dc9dace"
format_stamp: "Formatted at 2026-08-09 15:36:02 on dist-test-slave-2w3w"
server_key: "d15f4048bac07701997489e9248f07b9"
server_key_iv: "9fb25b4916ec49bbbafd466cbbf7f31d"
server_key_version: "encryptionkey@0"
I20260809 15:36:02.896095 9506 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.004s sys 0.000s
I20260809 15:36:02.898435 9521 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 15:36:02.899708 9506 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260809 15:36:02.899850 9506 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786289732823074-3431-0/minicluster-data/ts-0/data,/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786289732823074-3431-0/minicluster-data/ts-0/wal
uuid: "4b3563c4e6e749448e98a4042dc9dace"
format_stamp: "Formatted at 2026-08-09 15:36:02 on dist-test-slave-2w3w"
server_key: "d15f4048bac07701997489e9248f07b9"
server_key_iv: "9fb25b4916ec49bbbafd466cbbf7f31d"
server_key_version: "encryptionkey@0"
I20260809 15:36:02.899936 9506 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786289732823074-3431-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786289732823074-3431-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786289732823074-3431-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 15:36:02.922315 9506 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 15:36:02.925081 9506 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 15:36:02.925230 9506 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 15:36:02.925736 9506 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 15:36:02.926502 9506 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 15:36:02.926546 9506 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 15:36:02.926585 9506 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 15:36:02.926608 9506 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 15:36:02.935580 9506 rpc_server.cc:307] RPC server started. Bound to: 127.3.89.193:34555
I20260809 15:36:02.935658 9642 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.3.89.193:34555 every 8 connection(s)
I20260809 15:36:02.936576 9506 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786289732823074-3431-0/minicluster-data/ts-0/data/info.pb
I20260809 15:36:02.938509 3431 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskm314YR/build/debug/bin/kudu as pid 9506
I20260809 15:36:02.938644 3431 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786289732823074-3431-0/minicluster-data/ts-0/wal/instance
I20260809 15:36:02.939010 3431 external_mini_cluster.cc:1428] Setting key fb756a6290ea5d2bb35ea3c30ea52d93
Aug 09 15:36:02 dist-test-slave-2w3w krb5kdc[9389](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786289762, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.89.193@KRBTEST.COM for kudu/127.3.89.254@KRBTEST.COM
WARNING: no policy specified for kudu/127.3.89.194@KRBTEST.COM; defaulting to no policy
I20260809 15:36:02.952291 9482 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:36:02.938457 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.89.193:60509 (local address 127.3.89.254:45161)
0809 15:36:02.938639 (+ 182us) server_negotiation.cc:206] Beginning negotiation
0809 15:36:02.938645 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 15:36:02.939593 (+ 948us) server_negotiation.cc:407] Connection header received
0809 15:36:02.941166 (+ 1573us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 15:36:02.941170 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 15:36:02.941214 (+ 44us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 15:36:02.941302 (+ 88us) server_negotiation.cc:226] Negotiated authn=SASL
0809 15:36:02.942930 (+ 1628us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:36:02.943519 (+ 589us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:36:02.945129 (+ 1610us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:36:02.945399 (+ 270us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:36:02.945500 (+ 101us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:36:02.948528 (+ 3028us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 15:36:02.948532 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 15:36:02.948537 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 15:36:02.948570 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0809 15:36:02.950135 (+ 1565us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 15:36:02.950679 (+ 544us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 15:36:02.950683 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 15:36:02.950685 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 15:36:02.950795 (+ 110us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 15:36:02.951258 (+ 463us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 15:36:02.951261 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 15:36:02.951262 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 15:36:02.951444 (+ 182us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 15:36:02.951536 (+ 92us) server_negotiation.cc:1035] Waiting for connection context
0809 15:36:02.952061 (+ 525us) server_negotiation.cc:299] Negotiation successful
0809 15:36:02.952170 (+ 109us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":66}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.3.89.194@KRBTEST.COM" created.
I20260809 15:36:02.953368 9645 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:36:02.938894 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.89.254:45161 (local address 127.3.89.193:60509)
0809 15:36:02.939437 (+ 543us) negotiation.cc:107] Waiting for socket to connect
0809 15:36:02.939487 (+ 50us) client_negotiation.cc:174] Beginning negotiation
0809 15:36:02.940938 (+ 1451us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 15:36:02.941494 (+ 556us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 15:36:02.941506 (+ 12us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 15:36:02.942029 (+ 523us) client_negotiation.cc:189] Negotiated authn=SASL
0809 15:36:02.942770 (+ 741us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:36:02.942796 (+ 26us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:36:02.943655 (+ 859us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:36:02.943659 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:36:02.944984 (+ 1325us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:36:02.944998 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:36:02.945585 (+ 587us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:36:02.945593 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:36:02.945822 (+ 229us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:36:02.946507 (+ 685us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 15:36:02.946533 (+ 26us) client_negotiation.cc:656] calling sasl_client_start()
0809 15:36:02.948344 (+ 1811us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 15:36:02.950264 (+ 1920us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 15:36:02.950270 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 15:36:02.950285 (+ 15us) client_negotiation.cc:760] calling sasl_client_step()
0809 15:36:02.950557 (+ 272us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 15:36:02.950893 (+ 336us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 15:36:02.950898 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 15:36:02.950902 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0809 15:36:02.951150 (+ 248us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 15:36:02.951574 (+ 424us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 15:36:02.951579 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 15:36:02.951835 (+ 256us) client_negotiation.cc:769] Sending connection context
0809 15:36:02.952001 (+ 166us) client_negotiation.cc:240] Negotiation successful
0809 15:36:02.952338 (+ 337us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":321,"thread_start_us":161,"threads_started":1}
I20260809 15:36:02.954644 9643 heartbeater.cc:347] Connected to a master server at 127.3.89.254:45161
I20260809 15:36:02.954941 9643 heartbeater.cc:464] Registering TS with master...
I20260809 15:36:02.955571 9643 heartbeater.cc:511] Master 127.3.89.254:45161 requested a full tablet report, sending...
I20260809 15:36:02.957352 9439 ts_manager.cc:194] Registered new tserver with Master: 4b3563c4e6e749448e98a4042dc9dace (127.3.89.193:34555)
I20260809 15:36:02.959203 9439 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.3.89.193@KRBTEST.COM'} at 127.3.89.193:60509
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.3.89.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786289732823074-3431-0/krb5kdc/kudu_127.3.89.194.keytab.
Entry for principal kudu/127.3.89.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786289732823074-3431-0/krb5kdc/kudu_127.3.89.194.keytab.
I20260809 15:36:02.970409 3431 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskm314YR/build/debug/bin/kudu
/tmp/dist-test-taskm314YR/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786289732823074-3431-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786289732823074-3431-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786289732823074-3431-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786289732823074-3431-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.3.89.194:0
--local_ip_for_outbound_sockets=127.3.89.194
--webserver_interface=127.3.89.194
--webserver_port=0
--tserver_master_addrs=127.3.89.254:45161
--keytab_file=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786289732823074-3431-0/krb5kdc/kudu_127.3.89.194.keytab
--principal=kudu/127.3.89.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.3.89.212:42519
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786289732823074-3431-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786289732823074-3431-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786289732823074-3431-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 15:36:03.083608 9649 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 15:36:03.083873 9649 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 15:36:03.083916 9649 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 15:36:03.087891 9649 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 15:36:03.087947 9649 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 15:36:03.088008 9649 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.3.89.194
I20260809 15:36:03.093145 9649 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.3.89.212:42519
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786289732823074-3431-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786289732823074-3431-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786289732823074-3431-0/krb5kdc/kudu_127.3.89.194.keytab
--principal=kudu/127.3.89.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.3.89.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786289732823074-3431-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.3.89.194
--webserver_port=0
--tserver_master_addrs=127.3.89.254:45161
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.9649
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.3.89.194
--log_dir=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786289732823074-3431-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 09 Aug 2026 14:46:55 UTC on ccycloud.quasar-yswkmf.root.comops.site
build id 4576540
I20260809 15:36:03.094094 9649 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 15:36:03.095171 9649 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 15:36:03.104818 9657 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 15:36:03.104933 9649 server_base.cc:1034] running on GCE node
W20260809 15:36:03.104821 9655 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 15:36:03.104882 9654 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 15:36:03.105513 9649 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 15:36:03.106069 9649 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 15:36:03.107240 9649 hybrid_clock.cc:648] HybridClock initialized: now 1786289763107225 us; error 25 us; skew 500 ppm
Aug 09 15:36:03 dist-test-slave-2w3w krb5kdc[9389](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786289763, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.89.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 15:36:03.109922 9649 init.cc:377] Logged in from keytab as kudu/127.3.89.194@KRBTEST.COM (short username kudu)
I20260809 15:36:03.110710 9649 webserver.cc:460] Webserver started at http://127.3.89.194:34829/ using document root <none> and password file <none>
I20260809 15:36:03.111299 9649 fs_manager.cc:359] Metadata directory not provided
I20260809 15:36:03.111362 9649 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 15:36:03.111522 9649 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 15:36:03.113503 9649 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786289732823074-3431-0/minicluster-data/ts-1/data/instance:
uuid: "936e8a23d54c421fb0a4066b69e25693"
format_stamp: "Formatted at 2026-08-09 15:36:03 on dist-test-slave-2w3w"
server_key: "12161370dfa4da29c787e288092b995a"
server_key_iv: "926078ed0241602bc00f76d0c3aa195d"
server_key_version: "encryptionkey@0"
I20260809 15:36:03.114012 9649 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786289732823074-3431-0/minicluster-data/ts-1/wal/instance:
uuid: "936e8a23d54c421fb0a4066b69e25693"
format_stamp: "Formatted at 2026-08-09 15:36:03 on dist-test-slave-2w3w"
server_key: "12161370dfa4da29c787e288092b995a"
server_key_iv: "926078ed0241602bc00f76d0c3aa195d"
server_key_version: "encryptionkey@0"
I20260809 15:36:03.117717 9649 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.002s sys 0.004s
I20260809 15:36:03.119994 9664 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 15:36:03.121223 9649 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260809 15:36:03.121358 9649 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786289732823074-3431-0/minicluster-data/ts-1/data,/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786289732823074-3431-0/minicluster-data/ts-1/wal
uuid: "936e8a23d54c421fb0a4066b69e25693"
format_stamp: "Formatted at 2026-08-09 15:36:03 on dist-test-slave-2w3w"
server_key: "12161370dfa4da29c787e288092b995a"
server_key_iv: "926078ed0241602bc00f76d0c3aa195d"
server_key_version: "encryptionkey@0"
I20260809 15:36:03.121459 9649 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786289732823074-3431-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786289732823074-3431-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786289732823074-3431-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 15:36:03.135602 9649 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 15:36:03.138764 9649 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 15:36:03.138998 9649 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 15:36:03.139575 9649 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 15:36:03.140377 9649 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 15:36:03.140430 9649 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 15:36:03.140499 9649 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 15:36:03.140539 9649 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 15:36:03.148967 9649 rpc_server.cc:307] RPC server started. Bound to: 127.3.89.194:34771
I20260809 15:36:03.148999 9789 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.3.89.194:34771 every 8 connection(s)
I20260809 15:36:03.149833 9649 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786289732823074-3431-0/minicluster-data/ts-1/data/info.pb
I20260809 15:36:03.157425 3431 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskm314YR/build/debug/bin/kudu as pid 9649
I20260809 15:36:03.157558 3431 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786289732823074-3431-0/minicluster-data/ts-1/wal/instance
I20260809 15:36:03.157853 3431 external_mini_cluster.cc:1428] Setting key 383c395af58ef003edadc8a22301b370
Aug 09 15:36:03 dist-test-slave-2w3w krb5kdc[9389](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786289763, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.89.194@KRBTEST.COM for kudu/127.3.89.254@KRBTEST.COM
I20260809 15:36:03.164027 9482 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:36:03.151592 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.89.194:36721 (local address 127.3.89.254:45161)
0809 15:36:03.151725 (+ 133us) server_negotiation.cc:206] Beginning negotiation
0809 15:36:03.151729 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 15:36:03.152561 (+ 832us) server_negotiation.cc:407] Connection header received
0809 15:36:03.153923 (+ 1362us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 15:36:03.153926 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 15:36:03.154006 (+ 80us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 15:36:03.154100 (+ 94us) server_negotiation.cc:226] Negotiated authn=SASL
0809 15:36:03.155291 (+ 1191us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:36:03.155826 (+ 535us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:36:03.156966 (+ 1140us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:36:03.157188 (+ 222us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:36:03.157300 (+ 112us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:36:03.160017 (+ 2717us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 15:36:03.160023 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 15:36:03.160029 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 15:36:03.160062 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0809 15:36:03.161777 (+ 1715us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 15:36:03.162412 (+ 635us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 15:36:03.162414 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 15:36:03.162416 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 15:36:03.162494 (+ 78us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 15:36:03.162893 (+ 399us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 15:36:03.162895 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 15:36:03.162896 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 15:36:03.163119 (+ 223us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 15:36:03.163198 (+ 79us) server_negotiation.cc:1035] Waiting for connection context
0809 15:36:03.163803 (+ 605us) server_negotiation.cc:299] Negotiation successful
0809 15:36:03.163898 (+ 95us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":50}
I20260809 15:36:03.165227 9794 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:36:03.151974 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.89.254:45161 (local address 127.3.89.194:36721)
0809 15:36:03.152396 (+ 422us) negotiation.cc:107] Waiting for socket to connect
0809 15:36:03.152437 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0809 15:36:03.153706 (+ 1269us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 15:36:03.154262 (+ 556us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 15:36:03.154269 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 15:36:03.154614 (+ 345us) client_negotiation.cc:189] Negotiated authn=SASL
0809 15:36:03.155149 (+ 535us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:36:03.155164 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:36:03.155952 (+ 788us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:36:03.155964 (+ 12us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:36:03.156829 (+ 865us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:36:03.156838 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:36:03.157336 (+ 498us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:36:03.157340 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:36:03.157507 (+ 167us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:36:03.158179 (+ 672us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 15:36:03.158194 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0809 15:36:03.159843 (+ 1649us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 15:36:03.161909 (+ 2066us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 15:36:03.161918 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 15:36:03.161933 (+ 15us) client_negotiation.cc:760] calling sasl_client_step()
0809 15:36:03.162301 (+ 368us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 15:36:03.162583 (+ 282us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 15:36:03.162589 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 15:36:03.162593 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0809 15:36:03.162794 (+ 201us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 15:36:03.163226 (+ 432us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 15:36:03.163235 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 15:36:03.163547 (+ 312us) client_negotiation.cc:769] Sending connection context
0809 15:36:03.163758 (+ 211us) client_negotiation.cc:240] Negotiation successful
0809 15:36:03.164126 (+ 368us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":254,"thread_start_us":114,"threads_started":1}
I20260809 15:36:03.166406 9790 heartbeater.cc:347] Connected to a master server at 127.3.89.254:45161
I20260809 15:36:03.166649 9790 heartbeater.cc:464] Registering TS with master...
I20260809 15:36:03.167160 9790 heartbeater.cc:511] Master 127.3.89.254:45161 requested a full tablet report, sending...
I20260809 15:36:03.168211 9439 ts_manager.cc:194] Registered new tserver with Master: 936e8a23d54c421fb0a4066b69e25693 (127.3.89.194:34771)
I20260809 15:36:03.168812 9439 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.3.89.194@KRBTEST.COM'} at 127.3.89.194:36721
WARNING: no policy specified for kudu/127.3.89.195@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.3.89.195@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.3.89.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786289732823074-3431-0/krb5kdc/kudu_127.3.89.195.keytab.
Entry for principal kudu/127.3.89.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786289732823074-3431-0/krb5kdc/kudu_127.3.89.195.keytab.
I20260809 15:36:03.188112 3431 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskm314YR/build/debug/bin/kudu
/tmp/dist-test-taskm314YR/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786289732823074-3431-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786289732823074-3431-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786289732823074-3431-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786289732823074-3431-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.3.89.195:0
--local_ip_for_outbound_sockets=127.3.89.195
--webserver_interface=127.3.89.195
--webserver_port=0
--tserver_master_addrs=127.3.89.254:45161
--keytab_file=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786289732823074-3431-0/krb5kdc/kudu_127.3.89.195.keytab
--principal=kudu/127.3.89.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.3.89.212:42519
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786289732823074-3431-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786289732823074-3431-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786289732823074-3431-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 15:36:03.297082 9798 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 15:36:03.297389 9798 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 15:36:03.297464 9798 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 15:36:03.301725 9798 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 15:36:03.301816 9798 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 15:36:03.301904 9798 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.3.89.195
I20260809 15:36:03.307402 9798 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.3.89.212:42519
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786289732823074-3431-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786289732823074-3431-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786289732823074-3431-0/krb5kdc/kudu_127.3.89.195.keytab
--principal=kudu/127.3.89.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.3.89.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786289732823074-3431-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.3.89.195
--webserver_port=0
--tserver_master_addrs=127.3.89.254:45161
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.9798
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.3.89.195
--log_dir=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786289732823074-3431-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 09 Aug 2026 14:46:55 UTC on ccycloud.quasar-yswkmf.root.comops.site
build id 4576540
I20260809 15:36:03.308447 9798 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 15:36:03.309453 9798 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 15:36:03.318794 9805 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 15:36:03.318849 9807 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 15:36:03.318822 9804 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 15:36:03.319027 9798 server_base.cc:1034] running on GCE node
I20260809 15:36:03.319561 9798 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 15:36:03.320182 9798 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 15:36:03.321364 9798 hybrid_clock.cc:648] HybridClock initialized: now 1786289763321338 us; error 48 us; skew 500 ppm
Aug 09 15:36:03 dist-test-slave-2w3w krb5kdc[9389](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786289763, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.89.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 15:36:03.324213 9798 init.cc:377] Logged in from keytab as kudu/127.3.89.195@KRBTEST.COM (short username kudu)
I20260809 15:36:03.325013 9798 webserver.cc:460] Webserver started at http://127.3.89.195:39473/ using document root <none> and password file <none>
I20260809 15:36:03.325572 9798 fs_manager.cc:359] Metadata directory not provided
I20260809 15:36:03.325632 9798 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 15:36:03.325798 9798 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 15:36:03.327874 9798 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786289732823074-3431-0/minicluster-data/ts-2/data/instance:
uuid: "9e135dd4b0ee42a59d373ea08e17ce8f"
format_stamp: "Formatted at 2026-08-09 15:36:03 on dist-test-slave-2w3w"
server_key: "99fd329f18020e7cba38ea7ee23de8fc"
server_key_iv: "ab470aef5880c477e15a96056fdaa9be"
server_key_version: "encryptionkey@0"
I20260809 15:36:03.328454 9798 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786289732823074-3431-0/minicluster-data/ts-2/wal/instance:
uuid: "9e135dd4b0ee42a59d373ea08e17ce8f"
format_stamp: "Formatted at 2026-08-09 15:36:03 on dist-test-slave-2w3w"
server_key: "99fd329f18020e7cba38ea7ee23de8fc"
server_key_iv: "ab470aef5880c477e15a96056fdaa9be"
server_key_version: "encryptionkey@0"
I20260809 15:36:03.332154 9798 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.004s sys 0.000s
I20260809 15:36:03.334520 9814 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 15:36:03.335824 9798 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260809 15:36:03.335949 9798 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786289732823074-3431-0/minicluster-data/ts-2/data,/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786289732823074-3431-0/minicluster-data/ts-2/wal
uuid: "9e135dd4b0ee42a59d373ea08e17ce8f"
format_stamp: "Formatted at 2026-08-09 15:36:03 on dist-test-slave-2w3w"
server_key: "99fd329f18020e7cba38ea7ee23de8fc"
server_key_iv: "ab470aef5880c477e15a96056fdaa9be"
server_key_version: "encryptionkey@0"
I20260809 15:36:03.336052 9798 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786289732823074-3431-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786289732823074-3431-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786289732823074-3431-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 15:36:03.359992 9798 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 15:36:03.363300 9798 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 15:36:03.363507 9798 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 15:36:03.364144 9798 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 15:36:03.364974 9798 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 15:36:03.365046 9798 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 15:36:03.365119 9798 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 15:36:03.365163 9798 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 15:36:03.373872 9798 rpc_server.cc:307] RPC server started. Bound to: 127.3.89.195:36589
I20260809 15:36:03.373895 9941 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.3.89.195:36589 every 8 connection(s)
I20260809 15:36:03.374742 9798 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786289732823074-3431-0/minicluster-data/ts-2/data/info.pb
Aug 09 15:36:03 dist-test-slave-2w3w krb5kdc[9389](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786289763, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.89.195@KRBTEST.COM for kudu/127.3.89.254@KRBTEST.COM
I20260809 15:36:03.384649 3431 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskm314YR/build/debug/bin/kudu as pid 9798
I20260809 15:36:03.384801 3431 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786289732823074-3431-0/minicluster-data/ts-2/wal/instance
I20260809 15:36:03.385084 3431 external_mini_cluster.cc:1428] Setting key b3d718b5322824569012c054c817c2d6
I20260809 15:36:03.388586 9482 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:36:03.376727 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.89.195:39507 (local address 127.3.89.254:45161)
0809 15:36:03.376942 (+ 215us) server_negotiation.cc:206] Beginning negotiation
0809 15:36:03.376947 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 15:36:03.377594 (+ 647us) server_negotiation.cc:407] Connection header received
0809 15:36:03.378896 (+ 1302us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 15:36:03.378899 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 15:36:03.378948 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 15:36:03.379088 (+ 140us) server_negotiation.cc:226] Negotiated authn=SASL
0809 15:36:03.380170 (+ 1082us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:36:03.380742 (+ 572us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:36:03.381864 (+ 1122us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:36:03.382073 (+ 209us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:36:03.382150 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:36:03.384761 (+ 2611us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 15:36:03.384767 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 15:36:03.384773 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 15:36:03.384810 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0809 15:36:03.386602 (+ 1792us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 15:36:03.387208 (+ 606us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 15:36:03.387215 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 15:36:03.387220 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0809 15:36:03.387322 (+ 102us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 15:36:03.387685 (+ 363us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 15:36:03.387687 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 15:36:03.387689 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 15:36:03.387855 (+ 166us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 15:36:03.387932 (+ 77us) server_negotiation.cc:1035] Waiting for connection context
0809 15:36:03.388367 (+ 435us) server_negotiation.cc:299] Negotiation successful
0809 15:36:03.388468 (+ 101us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":111}
I20260809 15:36:03.389433 9945 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:36:03.376966 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.89.254:45161 (local address 127.3.89.195:39507)
0809 15:36:03.377443 (+ 477us) negotiation.cc:107] Waiting for socket to connect
0809 15:36:03.377482 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0809 15:36:03.378704 (+ 1222us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 15:36:03.379218 (+ 514us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 15:36:03.379224 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 15:36:03.379580 (+ 356us) client_negotiation.cc:189] Negotiated authn=SASL
0809 15:36:03.380034 (+ 454us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:36:03.380046 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:36:03.380888 (+ 842us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:36:03.380891 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:36:03.381740 (+ 849us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:36:03.381762 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:36:03.382296 (+ 534us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:36:03.382300 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:36:03.382441 (+ 141us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:36:03.383040 (+ 599us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 15:36:03.383055 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0809 15:36:03.384595 (+ 1540us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 15:36:03.386705 (+ 2110us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 15:36:03.386711 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 15:36:03.386725 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0809 15:36:03.387096 (+ 371us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 15:36:03.387470 (+ 374us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 15:36:03.387472 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 15:36:03.387474 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 15:36:03.387604 (+ 130us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 15:36:03.387943 (+ 339us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 15:36:03.387948 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 15:36:03.388160 (+ 212us) client_negotiation.cc:769] Sending connection context
0809 15:36:03.388323 (+ 163us) client_negotiation.cc:240] Negotiation successful
0809 15:36:03.388563 (+ 240us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":291,"thread_start_us":120,"threads_started":1}
I20260809 15:36:03.390483 9942 heartbeater.cc:347] Connected to a master server at 127.3.89.254:45161
I20260809 15:36:03.390722 9942 heartbeater.cc:464] Registering TS with master...
I20260809 15:36:03.391254 9942 heartbeater.cc:511] Master 127.3.89.254:45161 requested a full tablet report, sending...
I20260809 15:36:03.392244 9439 ts_manager.cc:194] Registered new tserver with Master: 9e135dd4b0ee42a59d373ea08e17ce8f (127.3.89.195:36589)
I20260809 15:36:03.392833 9439 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.3.89.195@KRBTEST.COM'} at 127.3.89.195:39507
I20260809 15:36:03.399129 3431 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal test-admin with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786289732823074-3431-0/krb5kdc/test-admin.keytab.
Entry for principal test-admin with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786289732823074-3431-0/krb5kdc/test-admin.keytab.
Aug 09 15:36:03 dist-test-slave-2w3w krb5kdc[9389](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786289763, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 15:36:03.415199 3431 init.cc:377] Logged in from keytab as test-admin@KRBTEST.COM (short username test-admin)
W20260809 15:36:03.417945 9950 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 15:36:03.416540 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40604 (local address 127.3.89.193:34555)
0809 15:36:03.416856 (+ 316us) server_negotiation.cc:206] Beginning negotiation
0809 15:36:03.416872 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0809 15:36:03.416897 (+ 25us) server_negotiation.cc:407] Connection header received
0809 15:36:03.416948 (+ 51us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 15:36:03.416954 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 15:36:03.417148 (+ 194us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 15:36:03.417271 (+ 123us) server_negotiation.cc:226] Negotiated authn=SASL
0809 15:36:03.417760 (+ 489us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:40604: BlockingRecv error: recv got EOF from 127.0.0.1:40604 (error 108)
Metrics: {"server-negotiator.queue_time_us":212,"thread_start_us":109,"threads_started":1}
W20260809 15:36:03.418028 3431 init.cc:300] Could not find kerberos principal in credential cache '/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786289732823074-3431-0/krb5kdc/krb5cc' of type FILE
Aug 09 15:36:03 dist-test-slave-2w3w krb5kdc[9389](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786289763, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 15:36:03.419387 3431 init.cc:377] Logged in from keytab as test-admin@KRBTEST.COM (short username test-admin)
Aug 09 15:36:03 dist-test-slave-2w3w krb5kdc[9389](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786289763, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 15:36:03.420353 3431 init.cc:295] Successfully reacquired a new kerberos TGT
Aug 09 15:36:03 dist-test-slave-2w3w krb5kdc[9389](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786289763, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.3.89.193@KRBTEST.COM
I20260809 15:36:03.430402 9950 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:36:03.421246 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40608 (local address 127.3.89.193:34555)
0809 15:36:03.421377 (+ 131us) server_negotiation.cc:206] Beginning negotiation
0809 15:36:03.421381 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 15:36:03.421493 (+ 112us) server_negotiation.cc:407] Connection header received
0809 15:36:03.421646 (+ 153us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 15:36:03.421649 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 15:36:03.421698 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 15:36:03.421785 (+ 87us) server_negotiation.cc:226] Negotiated authn=SASL
0809 15:36:03.422327 (+ 542us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:36:03.423184 (+ 857us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:36:03.424033 (+ 849us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:36:03.424488 (+ 455us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:36:03.424604 (+ 116us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:36:03.427084 (+ 2480us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 15:36:03.427093 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 15:36:03.427107 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 15:36:03.427131 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0809 15:36:03.428705 (+ 1574us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 15:36:03.429134 (+ 429us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 15:36:03.429138 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 15:36:03.429140 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 15:36:03.429214 (+ 74us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 15:36:03.429544 (+ 330us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 15:36:03.429546 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 15:36:03.429547 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 15:36:03.429770 (+ 223us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 15:36:03.429926 (+ 156us) server_negotiation.cc:1035] Waiting for connection context
0809 15:36:03.430138 (+ 212us) server_negotiation.cc:299] Negotiation successful
0809 15:36:03.430262 (+ 124us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":52}
W20260809 15:36:03.434481 9950 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 15:36:03.433432 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40610 (local address 127.3.89.193:34555)
0809 15:36:03.433570 (+ 138us) server_negotiation.cc:206] Beginning negotiation
0809 15:36:03.433574 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 15:36:03.433678 (+ 104us) server_negotiation.cc:407] Connection header received
0809 15:36:03.433836 (+ 158us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 15:36:03.433838 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 15:36:03.433873 (+ 35us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 15:36:03.433954 (+ 81us) server_negotiation.cc:226] Negotiated authn=SASL
0809 15:36:03.434364 (+ 410us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:40610: BlockingRecv error: recv got EOF from 127.0.0.1:40610 (error 108)
Metrics: {"server-negotiator.queue_time_us":59}
W20260809 15:36:03.434616 3431 init.cc:300] Could not find kerberos principal in credential cache '/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786289732823074-3431-0/krb5kdc/krb5cc' of type FILE
Aug 09 15:36:03 dist-test-slave-2w3w krb5kdc[9389](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786289763, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 15:36:03.435714 3431 init.cc:377] Logged in from keytab as test-admin@KRBTEST.COM (short username test-admin)
Aug 09 15:36:03 dist-test-slave-2w3w krb5kdc[9389](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786289763, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 15:36:03.436718 3431 init.cc:295] Successfully reacquired a new kerberos TGT
Aug 09 15:36:03 dist-test-slave-2w3w krb5kdc[9389](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786289763, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.3.89.193@KRBTEST.COM
I20260809 15:36:03.445309 9950 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:36:03.437579 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40620 (local address 127.3.89.193:34555)
0809 15:36:03.437704 (+ 125us) server_negotiation.cc:206] Beginning negotiation
0809 15:36:03.437708 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 15:36:03.437887 (+ 179us) server_negotiation.cc:407] Connection header received
0809 15:36:03.438004 (+ 117us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 15:36:03.438007 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 15:36:03.438050 (+ 43us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 15:36:03.438132 (+ 82us) server_negotiation.cc:226] Negotiated authn=SASL
0809 15:36:03.438638 (+ 506us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:36:03.439252 (+ 614us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:36:03.440170 (+ 918us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:36:03.440401 (+ 231us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:36:03.440535 (+ 134us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:36:03.442681 (+ 2146us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 15:36:03.442684 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 15:36:03.442686 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 15:36:03.442707 (+ 21us) server_negotiation.cc:922] calling sasl_server_start()
0809 15:36:03.443929 (+ 1222us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 15:36:03.444313 (+ 384us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 15:36:03.444316 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 15:36:03.444317 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 15:36:03.444386 (+ 69us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 15:36:03.444613 (+ 227us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 15:36:03.444615 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 15:36:03.444616 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 15:36:03.444762 (+ 146us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 15:36:03.444857 (+ 95us) server_negotiation.cc:1035] Waiting for connection context
0809 15:36:03.445107 (+ 250us) server_negotiation.cc:299] Negotiation successful
0809 15:36:03.445198 (+ 91us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":42}
W20260809 15:36:03.448185 9950 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 15:36:03.447108 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40624 (local address 127.3.89.193:34555)
0809 15:36:03.447240 (+ 132us) server_negotiation.cc:206] Beginning negotiation
0809 15:36:03.447244 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 15:36:03.447368 (+ 124us) server_negotiation.cc:407] Connection header received
0809 15:36:03.447505 (+ 137us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 15:36:03.447509 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 15:36:03.447551 (+ 42us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 15:36:03.447634 (+ 83us) server_negotiation.cc:226] Negotiated authn=SASL
0809 15:36:03.448078 (+ 444us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:40624: BlockingRecv error: recv got EOF from 127.0.0.1:40624 (error 108)
Metrics: {"server-negotiator.queue_time_us":58}
W20260809 15:36:03.448357 3431 init.cc:300] Could not find kerberos principal in credential cache '/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786289732823074-3431-0/krb5kdc/krb5cc' of type FILE
Aug 09 15:36:03 dist-test-slave-2w3w krb5kdc[9389](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786289763, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 15:36:03.449345 3431 init.cc:377] Logged in from keytab as test-admin@KRBTEST.COM (short username test-admin)
Aug 09 15:36:03 dist-test-slave-2w3w krb5kdc[9389](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786289763, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 15:36:03.450196 3431 init.cc:295] Successfully reacquired a new kerberos TGT
Aug 09 15:36:03 dist-test-slave-2w3w krb5kdc[9389](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786289763, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.3.89.193@KRBTEST.COM
I20260809 15:36:03.457934 9950 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:36:03.450943 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40640 (local address 127.3.89.193:34555)
0809 15:36:03.451087 (+ 144us) server_negotiation.cc:206] Beginning negotiation
0809 15:36:03.451090 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0809 15:36:03.451194 (+ 104us) server_negotiation.cc:407] Connection header received
0809 15:36:03.451263 (+ 69us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 15:36:03.451265 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 15:36:03.451295 (+ 30us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 15:36:03.451363 (+ 68us) server_negotiation.cc:226] Negotiated authn=SASL
0809 15:36:03.451839 (+ 476us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:36:03.452294 (+ 455us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:36:03.453013 (+ 719us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:36:03.453212 (+ 199us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:36:03.453285 (+ 73us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:36:03.455371 (+ 2086us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 15:36:03.455374 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 15:36:03.455376 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 15:36:03.455396 (+ 20us) server_negotiation.cc:922] calling sasl_server_start()
0809 15:36:03.456593 (+ 1197us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 15:36:03.456988 (+ 395us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 15:36:03.456991 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 15:36:03.456992 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 15:36:03.457069 (+ 77us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 15:36:03.457306 (+ 237us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 15:36:03.457308 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 15:36:03.457309 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 15:36:03.457454 (+ 145us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 15:36:03.457548 (+ 94us) server_negotiation.cc:1035] Waiting for connection context
0809 15:36:03.457738 (+ 190us) server_negotiation.cc:299] Negotiation successful
0809 15:36:03.457832 (+ 94us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":80}
I20260809 15:36:03.458918 3431 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskm314YR/build/debug/bin/kudu with pid 9506
I20260809 15:36:03.466400 3431 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskm314YR/build/debug/bin/kudu with pid 9649
I20260809 15:36:03.473687 3431 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskm314YR/build/debug/bin/kudu with pid 9798
I20260809 15:36:03.481055 3431 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskm314YR/build/debug/bin/kudu with pid 9403
2026-08-09T15:36:03Z chronyd exiting
[ OK ] SecurityITest.TestCorruptKerberosCC (2266 ms)
[ RUN ] SecurityITest.TestNonDefaultPrincipalMultipleMaster
Loading random data
Initializing database '/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786289732823074-3431-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 15:36:03 dist-test-slave-2w3w krb5kdc[9964](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 15:36:03 dist-test-slave-2w3w krb5kdc[9964](info): set up 2 sockets
Aug 09 15:36:03 dist-test-slave-2w3w krb5kdc[9964](info): commencing operation
krb5kdc: starting...
W20260809 15:36:04.636868 3431 mini_kdc.cc:121] Time spent starting KDC: real 1.130s user 0.000s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786289732823074-3431-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786289732823074-3431-0/krb5kdc/test-user.keytab.
Aug 09 15:36:04 dist-test-slave-2w3w krb5kdc[9964](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786289764, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T15:36:04Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T15:36:04Z Disabled control of system clock
WARNING: no policy specified for oryx/127.3.89.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.3.89.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.3.89.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786289732823074-3431-0/krb5kdc/oryx_127.3.89.254.keytab.
Entry for principal oryx/127.3.89.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786289732823074-3431-0/krb5kdc/oryx_127.3.89.254.keytab.
I20260809 15:36:04.765792 3431 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskm314YR/build/debug/bin/kudu
/tmp/dist-test-taskm314YR/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786289732823074-3431-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786289732823074-3431-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786289732823074-3431-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786289732823074-3431-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.3.89.254:46611
--webserver_interface=127.3.89.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786289732823074-3431-0/krb5kdc/oryx_127.3.89.254.keytab
--principal=oryx/127.3.89.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.3.89.212:43259
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.3.89.254:46611
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786289732823074-3431-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786289732823074-3431-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786289732823074-3431-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 15:36:04.878121 9978 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 15:36:04.878441 9978 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 15:36:04.878523 9978 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 15:36:04.882522 9978 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 15:36:04.882620 9978 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 15:36:04.882663 9978 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 15:36:04.882699 9978 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 15:36:04.882728 9978 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 15:36:04.887887 9978 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.3.89.212:43259
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786289732823074-3431-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786289732823074-3431-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.3.89.254:46611
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786289732823074-3431-0/krb5kdc/oryx_127.3.89.254.keytab
--principal=oryx/127.3.89.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.3.89.254:46611
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786289732823074-3431-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.3.89.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.9978
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786289732823074-3431-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 09 Aug 2026 14:46:55 UTC on ccycloud.quasar-yswkmf.root.comops.site
build id 4576540
I20260809 15:36:04.888958 9978 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 15:36:04.889955 9978 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 15:36:04.898355 9984 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 15:36:04.898486 9985 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 15:36:04.898399 9978 server_base.cc:1034] running on GCE node
W20260809 15:36:04.898370 9987 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 15:36:04.899171 9978 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 15:36:04.900238 9978 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 15:36:04.901440 9978 hybrid_clock.cc:648] HybridClock initialized: now 1786289764901424 us; error 30 us; skew 500 ppm
Aug 09 15:36:04 dist-test-slave-2w3w krb5kdc[9964](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786289764, etypes {rep=17 tkt=17 ses=17}, oryx/127.3.89.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 15:36:04.909475 9978 init.cc:377] Logged in from keytab as oryx/127.3.89.254@KRBTEST.COM (short username oryx)
I20260809 15:36:04.910342 9978 webserver.cc:460] Webserver started at http://127.3.89.254:33585/ using document root <none> and password file <none>
I20260809 15:36:04.910856 9978 fs_manager.cc:359] Metadata directory not provided
I20260809 15:36:04.910914 9978 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 15:36:04.911144 9978 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 15:36:04.913154 9978 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786289732823074-3431-0/minicluster-data/master-0/data/instance:
uuid: "b7ef4cef863642f19bd9f01ae2792fe3"
format_stamp: "Formatted at 2026-08-09 15:36:04 on dist-test-slave-2w3w"
server_key: "422f61faf48707f281f79d75301e9035"
server_key_iv: "3e8bd01ded4b0d9a2b939180de9b6897"
server_key_version: "encryptionkey@0"
I20260809 15:36:04.913678 9978 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786289732823074-3431-0/minicluster-data/master-0/wal/instance:
uuid: "b7ef4cef863642f19bd9f01ae2792fe3"
format_stamp: "Formatted at 2026-08-09 15:36:04 on dist-test-slave-2w3w"
server_key: "422f61faf48707f281f79d75301e9035"
server_key_iv: "3e8bd01ded4b0d9a2b939180de9b6897"
server_key_version: "encryptionkey@0"
I20260809 15:36:04.917309 9978 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.001s sys 0.004s
I20260809 15:36:04.919636 9996 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 15:36:04.920814 9978 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260809 15:36:04.920949 9978 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786289732823074-3431-0/minicluster-data/master-0/data,/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786289732823074-3431-0/minicluster-data/master-0/wal
uuid: "b7ef4cef863642f19bd9f01ae2792fe3"
format_stamp: "Formatted at 2026-08-09 15:36:04 on dist-test-slave-2w3w"
server_key: "422f61faf48707f281f79d75301e9035"
server_key_iv: "3e8bd01ded4b0d9a2b939180de9b6897"
server_key_version: "encryptionkey@0"
I20260809 15:36:04.921058 9978 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786289732823074-3431-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786289732823074-3431-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786289732823074-3431-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 15:36:04.953109 9978 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 15:36:04.956825 9978 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 15:36:04.957060 9978 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 15:36:04.964408 9978 rpc_server.cc:307] RPC server started. Bound to: 127.3.89.254:46611
I20260809 15:36:04.964695 10052 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.3.89.254:46611 every 8 connection(s)
I20260809 15:36:04.965441 9978 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786289732823074-3431-0/minicluster-data/master-0/data/info.pb
I20260809 15:36:04.968165 10053 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 15:36:04.973541 3431 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskm314YR/build/debug/bin/kudu as pid 9978
I20260809 15:36:04.973546 10053 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b7ef4cef863642f19bd9f01ae2792fe3: Bootstrap starting.
I20260809 15:36:04.973691 3431 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786289732823074-3431-0/minicluster-data/master-0/wal/instance
I20260809 15:36:04.974025 3431 external_mini_cluster.cc:1428] Setting key 68054bd0dead2dd8abddb75f1a34ba1f
I20260809 15:36:04.976215 10053 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P b7ef4cef863642f19bd9f01ae2792fe3: Neither blocks nor log segments found. Creating new log.
I20260809 15:36:04.976938 10053 log.cc:824] T 00000000000000000000000000000000 P b7ef4cef863642f19bd9f01ae2792fe3: Log is configured to *not* fsync() on all Append() calls
I20260809 15:36:04.978739 10053 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b7ef4cef863642f19bd9f01ae2792fe3: No bootstrap required, opened a new log
I20260809 15:36:04.982054 10053 raft_consensus.cc:348] T 00000000000000000000000000000000 P b7ef4cef863642f19bd9f01ae2792fe3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b7ef4cef863642f19bd9f01ae2792fe3" member_type: VOTER last_known_addr { host: "127.3.89.254" port: 46611 } }
I20260809 15:36:04.982261 10053 raft_consensus.cc:374] T 00000000000000000000000000000000 P b7ef4cef863642f19bd9f01ae2792fe3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 15:36:04.982337 10053 raft_consensus.cc:729] T 00000000000000000000000000000000 P b7ef4cef863642f19bd9f01ae2792fe3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b7ef4cef863642f19bd9f01ae2792fe3, State: Initialized, Role: FOLLOWER
Aug 09 15:36:04 dist-test-slave-2w3w krb5kdc[9964](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786289764, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.3.89.254@KRBTEST.COM
I20260809 15:36:04.982743 10053 consensus_queue.cc:260] T 00000000000000000000000000000000 P b7ef4cef863642f19bd9f01ae2792fe3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b7ef4cef863642f19bd9f01ae2792fe3" member_type: VOTER last_known_addr { host: "127.3.89.254" port: 46611 } }
I20260809 15:36:04.982879 10053 raft_consensus.cc:388] T 00000000000000000000000000000000 P b7ef4cef863642f19bd9f01ae2792fe3 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 15:36:04.982945 10053 raft_consensus.cc:482] T 00000000000000000000000000000000 P b7ef4cef863642f19bd9f01ae2792fe3 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 15:36:04.983081 10053 raft_consensus.cc:3037] T 00000000000000000000000000000000 P b7ef4cef863642f19bd9f01ae2792fe3 [term 0 FOLLOWER]: Advancing to term 1
I20260809 15:36:04.984280 10053 raft_consensus.cc:504] T 00000000000000000000000000000000 P b7ef4cef863642f19bd9f01ae2792fe3 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b7ef4cef863642f19bd9f01ae2792fe3" member_type: VOTER last_known_addr { host: "127.3.89.254" port: 46611 } }
I20260809 15:36:04.984675 10053 leader_election.cc:302] T 00000000000000000000000000000000 P b7ef4cef863642f19bd9f01ae2792fe3 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: b7ef4cef863642f19bd9f01ae2792fe3; no voters:
I20260809 15:36:04.985064 10053 leader_election.cc:288] T 00000000000000000000000000000000 P b7ef4cef863642f19bd9f01ae2792fe3 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 15:36:04.985173 10059 raft_consensus.cc:2781] T 00000000000000000000000000000000 P b7ef4cef863642f19bd9f01ae2792fe3 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 15:36:04.985421 10059 raft_consensus.cc:686] T 00000000000000000000000000000000 P b7ef4cef863642f19bd9f01ae2792fe3 [term 1 LEADER]: Becoming Leader. State: Replica: b7ef4cef863642f19bd9f01ae2792fe3, State: Running, Role: LEADER
I20260809 15:36:04.985785 10059 consensus_queue.cc:237] T 00000000000000000000000000000000 P b7ef4cef863642f19bd9f01ae2792fe3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b7ef4cef863642f19bd9f01ae2792fe3" member_type: VOTER last_known_addr { host: "127.3.89.254" port: 46611 } }
I20260809 15:36:04.986387 10053 sys_catalog.cc:564] T 00000000000000000000000000000000 P b7ef4cef863642f19bd9f01ae2792fe3 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 15:36:04.986936 10059 sys_catalog.cc:455] T 00000000000000000000000000000000 P b7ef4cef863642f19bd9f01ae2792fe3 [sys.catalog]: SysCatalogTable state changed. Reason: New leader b7ef4cef863642f19bd9f01ae2792fe3. Latest consensus state: current_term: 1 leader_uuid: "b7ef4cef863642f19bd9f01ae2792fe3" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b7ef4cef863642f19bd9f01ae2792fe3" member_type: VOTER last_known_addr { host: "127.3.89.254" port: 46611 } } }
I20260809 15:36:04.987140 10059 sys_catalog.cc:458] T 00000000000000000000000000000000 P b7ef4cef863642f19bd9f01ae2792fe3 [sys.catalog]: This master's current role is: LEADER
I20260809 15:36:04.987401 10061 sys_catalog.cc:455] T 00000000000000000000000000000000 P b7ef4cef863642f19bd9f01ae2792fe3 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "b7ef4cef863642f19bd9f01ae2792fe3" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b7ef4cef863642f19bd9f01ae2792fe3" member_type: VOTER last_known_addr { host: "127.3.89.254" port: 46611 } } }
I20260809 15:36:04.987504 10061 sys_catalog.cc:458] T 00000000000000000000000000000000 P b7ef4cef863642f19bd9f01ae2792fe3 [sys.catalog]: This master's current role is: LEADER
I20260809 15:36:04.987641 10067 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 15:36:04.989686 10057 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:36:04.975406 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53114 (local address 127.3.89.254:46611)
0809 15:36:04.975818 (+ 412us) server_negotiation.cc:206] Beginning negotiation
0809 15:36:04.975834 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0809 15:36:04.975869 (+ 35us) server_negotiation.cc:407] Connection header received
0809 15:36:04.976553 (+ 684us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 15:36:04.976571 (+ 18us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 15:36:04.976891 (+ 320us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 15:36:04.977252 (+ 361us) server_negotiation.cc:226] Negotiated authn=SASL
0809 15:36:04.978063 (+ 811us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:36:04.979191 (+ 1128us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:36:04.980029 (+ 838us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:36:04.980507 (+ 478us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:36:04.980670 (+ 163us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:36:04.983482 (+ 2812us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 15:36:04.983488 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 15:36:04.983503 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 15:36:04.983536 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0809 15:36:04.985655 (+ 2119us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 15:36:04.986197 (+ 542us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 15:36:04.986202 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 15:36:04.986208 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0809 15:36:04.986340 (+ 132us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 15:36:04.986659 (+ 319us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 15:36:04.986662 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 15:36:04.986664 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 15:36:04.987289 (+ 625us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 15:36:04.988158 (+ 869us) server_negotiation.cc:1035] Waiting for connection context
0809 15:36:04.988338 (+ 180us) server_negotiation.cc:299] Negotiation successful
0809 15:36:04.988664 (+ 326us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":262,"thread_start_us":140,"threads_started":1}
I20260809 15:36:04.991586 10067 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 15:36:04.996766 10067 catalog_manager.cc:1347] Generated new cluster ID: 1422be81015249aead82021a784a5898
I20260809 15:36:04.996829 10067 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 15:36:05.012305 10067 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 15:36:05.013635 10067 catalog_manager.cc:1504] Loading token signing keys...
I20260809 15:36:05.025413 10067 catalog_manager.cc:5868] T 00000000000000000000000000000000 P b7ef4cef863642f19bd9f01ae2792fe3: Generated new TSK 0
I20260809 15:36:05.026329 10067 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for oryx/127.3.89.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.3.89.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.3.89.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786289732823074-3431-0/krb5kdc/oryx_127.3.89.193.keytab.
Entry for principal oryx/127.3.89.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786289732823074-3431-0/krb5kdc/oryx_127.3.89.193.keytab.
I20260809 15:36:05.062703 3431 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskm314YR/build/debug/bin/kudu
/tmp/dist-test-taskm314YR/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786289732823074-3431-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786289732823074-3431-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786289732823074-3431-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786289732823074-3431-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.3.89.193:0
--local_ip_for_outbound_sockets=127.3.89.193
--webserver_interface=127.3.89.193
--webserver_port=0
--tserver_master_addrs=127.3.89.254:46611
--keytab_file=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786289732823074-3431-0/krb5kdc/oryx_127.3.89.193.keytab
--principal=oryx/127.3.89.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.3.89.212:43259
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786289732823074-3431-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786289732823074-3431-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786289732823074-3431-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 15:36:05.184777 10082 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 15:36:05.185107 10082 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 15:36:05.185189 10082 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 15:36:05.190083 10082 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 15:36:05.190191 10082 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 15:36:05.190294 10082 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.3.89.193
I20260809 15:36:05.196022 10082 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.3.89.212:43259
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786289732823074-3431-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786289732823074-3431-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786289732823074-3431-0/krb5kdc/oryx_127.3.89.193.keytab
--principal=oryx/127.3.89.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.3.89.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786289732823074-3431-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.3.89.193
--webserver_port=0
--tserver_master_addrs=127.3.89.254:46611
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.10082
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.3.89.193
--log_dir=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786289732823074-3431-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 09 Aug 2026 14:46:55 UTC on ccycloud.quasar-yswkmf.root.comops.site
build id 4576540
I20260809 15:36:05.197206 10082 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 15:36:05.198258 10082 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 15:36:05.208079 10087 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 15:36:05.208190 10082 server_base.cc:1034] running on GCE node
W20260809 15:36:05.208081 10088 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 15:36:05.208077 10091 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 15:36:05.209425 10082 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 15:36:05.210080 10082 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 15:36:05.211249 10082 hybrid_clock.cc:648] HybridClock initialized: now 1786289765211209 us; error 52 us; skew 500 ppm
Aug 09 15:36:05 dist-test-slave-2w3w krb5kdc[9964](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786289765, etypes {rep=17 tkt=17 ses=17}, oryx/127.3.89.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 15:36:05.214149 10082 init.cc:377] Logged in from keytab as oryx/127.3.89.193@KRBTEST.COM (short username oryx)
I20260809 15:36:05.215167 10082 webserver.cc:460] Webserver started at http://127.3.89.193:34961/ using document root <none> and password file <none>
I20260809 15:36:05.215718 10082 fs_manager.cc:359] Metadata directory not provided
I20260809 15:36:05.215790 10082 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 15:36:05.216027 10082 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 15:36:05.218050 10082 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786289732823074-3431-0/minicluster-data/ts-0/data/instance:
uuid: "5cbab4c3b10e42448c1d2d3cd71ea8be"
format_stamp: "Formatted at 2026-08-09 15:36:05 on dist-test-slave-2w3w"
server_key: "08a21654c0209b1aab0fdcab6b3594a4"
server_key_iv: "c75e0032f961a470d7d1e176549bf51f"
server_key_version: "encryptionkey@0"
I20260809 15:36:05.218618 10082 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786289732823074-3431-0/minicluster-data/ts-0/wal/instance:
uuid: "5cbab4c3b10e42448c1d2d3cd71ea8be"
format_stamp: "Formatted at 2026-08-09 15:36:05 on dist-test-slave-2w3w"
server_key: "08a21654c0209b1aab0fdcab6b3594a4"
server_key_iv: "c75e0032f961a470d7d1e176549bf51f"
server_key_version: "encryptionkey@0"
I20260809 15:36:05.222513 10082 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.006s sys 0.000s
I20260809 15:36:05.225059 10099 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 15:36:05.226306 10082 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260809 15:36:05.226424 10082 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786289732823074-3431-0/minicluster-data/ts-0/data,/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786289732823074-3431-0/minicluster-data/ts-0/wal
uuid: "5cbab4c3b10e42448c1d2d3cd71ea8be"
format_stamp: "Formatted at 2026-08-09 15:36:05 on dist-test-slave-2w3w"
server_key: "08a21654c0209b1aab0fdcab6b3594a4"
server_key_iv: "c75e0032f961a470d7d1e176549bf51f"
server_key_version: "encryptionkey@0"
I20260809 15:36:05.226523 10082 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786289732823074-3431-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786289732823074-3431-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786289732823074-3431-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 15:36:05.240149 10082 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 15:36:05.243031 10082 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 15:36:05.243242 10082 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 15:36:05.243826 10082 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 15:36:05.244626 10082 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 15:36:05.244699 10082 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 15:36:05.244844 10082 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 15:36:05.244910 10082 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 15:36:05.253631 10082 rpc_server.cc:307] RPC server started. Bound to: 127.3.89.193:44753
I20260809 15:36:05.253664 10222 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.3.89.193:44753 every 8 connection(s)
I20260809 15:36:05.254518 10082 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786289732823074-3431-0/minicluster-data/ts-0/data/info.pb
I20260809 15:36:05.260701 3431 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskm314YR/build/debug/bin/kudu as pid 10082
I20260809 15:36:05.260861 3431 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786289732823074-3431-0/minicluster-data/ts-0/wal/instance
I20260809 15:36:05.261188 3431 external_mini_cluster.cc:1428] Setting key 22883c7eea0ab1308125f681411fbe8e
Aug 09 15:36:05 dist-test-slave-2w3w krb5kdc[9964](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786289765, etypes {rep=17 tkt=17 ses=17}, oryx/127.3.89.193@KRBTEST.COM for oryx/127.3.89.254@KRBTEST.COM
I20260809 15:36:05.269307 10057 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:36:05.256432 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.89.193:44065 (local address 127.3.89.254:46611)
0809 15:36:05.256586 (+ 154us) server_negotiation.cc:206] Beginning negotiation
0809 15:36:05.256590 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 15:36:05.257392 (+ 802us) server_negotiation.cc:407] Connection header received
0809 15:36:05.258696 (+ 1304us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 15:36:05.258699 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 15:36:05.258757 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 15:36:05.258839 (+ 82us) server_negotiation.cc:226] Negotiated authn=SASL
0809 15:36:05.260006 (+ 1167us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:36:05.260551 (+ 545us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:36:05.262043 (+ 1492us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:36:05.262278 (+ 235us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:36:05.262388 (+ 110us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:36:05.265296 (+ 2908us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 15:36:05.265302 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 15:36:05.265306 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 15:36:05.265340 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0809 15:36:05.267196 (+ 1856us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 15:36:05.267691 (+ 495us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 15:36:05.267698 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 15:36:05.267703 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0809 15:36:05.267833 (+ 130us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 15:36:05.268203 (+ 370us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 15:36:05.268207 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 15:36:05.268208 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 15:36:05.268396 (+ 188us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 15:36:05.268505 (+ 109us) server_negotiation.cc:1035] Waiting for connection context
0809 15:36:05.268990 (+ 485us) server_negotiation.cc:299] Negotiation successful
0809 15:36:05.269117 (+ 127us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":65}
I20260809 15:36:05.270160 10225 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:36:05.256780 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.89.254:46611 (local address 127.3.89.193:44065)
0809 15:36:05.257223 (+ 443us) negotiation.cc:107] Waiting for socket to connect
0809 15:36:05.257270 (+ 47us) client_negotiation.cc:174] Beginning negotiation
0809 15:36:05.258479 (+ 1209us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 15:36:05.259033 (+ 554us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 15:36:05.259040 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 15:36:05.259385 (+ 345us) client_negotiation.cc:189] Negotiated authn=SASL
0809 15:36:05.259845 (+ 460us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:36:05.259858 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:36:05.260702 (+ 844us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:36:05.260705 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:36:05.261885 (+ 1180us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:36:05.261918 (+ 33us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:36:05.262439 (+ 521us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:36:05.262442 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:36:05.262665 (+ 223us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:36:05.263381 (+ 716us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 15:36:05.263398 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0809 15:36:05.265116 (+ 1718us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 15:36:05.267314 (+ 2198us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 15:36:05.267319 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 15:36:05.267329 (+ 10us) client_negotiation.cc:760] calling sasl_client_step()
0809 15:36:05.267581 (+ 252us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 15:36:05.267933 (+ 352us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 15:36:05.267935 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 15:36:05.267936 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0809 15:36:05.268074 (+ 138us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 15:36:05.268514 (+ 440us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 15:36:05.268523 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 15:36:05.268761 (+ 238us) client_negotiation.cc:769] Sending connection context
0809 15:36:05.268948 (+ 187us) client_negotiation.cc:240] Negotiation successful
0809 15:36:05.269264 (+ 316us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":266,"thread_start_us":124,"threads_started":1}
I20260809 15:36:05.271482 10223 heartbeater.cc:347] Connected to a master server at 127.3.89.254:46611
I20260809 15:36:05.271763 10223 heartbeater.cc:464] Registering TS with master...
WARNING: no policy specified for oryx/127.3.89.194@KRBTEST.COM; defaulting to no policy
I20260809 15:36:05.272331 10223 heartbeater.cc:511] Master 127.3.89.254:46611 requested a full tablet report, sending...
I20260809 15:36:05.273939 10014 ts_manager.cc:194] Registered new tserver with Master: 5cbab4c3b10e42448c1d2d3cd71ea8be (127.3.89.193:44753)
I20260809 15:36:05.275060 10014 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.3.89.193@KRBTEST.COM'} at 127.3.89.193:44065
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.3.89.194@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.3.89.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786289732823074-3431-0/krb5kdc/oryx_127.3.89.194.keytab.
Entry for principal oryx/127.3.89.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786289732823074-3431-0/krb5kdc/oryx_127.3.89.194.keytab.
I20260809 15:36:05.293706 3431 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskm314YR/build/debug/bin/kudu
/tmp/dist-test-taskm314YR/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786289732823074-3431-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786289732823074-3431-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786289732823074-3431-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786289732823074-3431-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.3.89.194:0
--local_ip_for_outbound_sockets=127.3.89.194
--webserver_interface=127.3.89.194
--webserver_port=0
--tserver_master_addrs=127.3.89.254:46611
--keytab_file=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786289732823074-3431-0/krb5kdc/oryx_127.3.89.194.keytab
--principal=oryx/127.3.89.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.3.89.212:43259
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786289732823074-3431-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786289732823074-3431-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786289732823074-3431-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 15:36:05.404299 10228 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 15:36:05.404609 10228 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 15:36:05.404655 10228 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 15:36:05.408717 10228 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 15:36:05.408777 10228 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 15:36:05.408867 10228 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.3.89.194
I20260809 15:36:05.414175 10228 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.3.89.212:43259
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786289732823074-3431-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786289732823074-3431-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786289732823074-3431-0/krb5kdc/oryx_127.3.89.194.keytab
--principal=oryx/127.3.89.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.3.89.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786289732823074-3431-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.3.89.194
--webserver_port=0
--tserver_master_addrs=127.3.89.254:46611
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.10228
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.3.89.194
--log_dir=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786289732823074-3431-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 09 Aug 2026 14:46:55 UTC on ccycloud.quasar-yswkmf.root.comops.site
build id 4576540
I20260809 15:36:05.415295 10228 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 15:36:05.416301 10228 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 15:36:05.425724 10236 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 15:36:05.425729 10238 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 15:36:05.425714 10235 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 15:36:05.426148 10228 server_base.cc:1034] running on GCE node
I20260809 15:36:05.426565 10228 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 15:36:05.427251 10228 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 15:36:05.428408 10228 hybrid_clock.cc:648] HybridClock initialized: now 1786289765428392 us; error 31 us; skew 500 ppm
Aug 09 15:36:05 dist-test-slave-2w3w krb5kdc[9964](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786289765, etypes {rep=17 tkt=17 ses=17}, oryx/127.3.89.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 15:36:05.431200 10228 init.cc:377] Logged in from keytab as oryx/127.3.89.194@KRBTEST.COM (short username oryx)
I20260809 15:36:05.432039 10228 webserver.cc:460] Webserver started at http://127.3.89.194:43137/ using document root <none> and password file <none>
I20260809 15:36:05.432602 10228 fs_manager.cc:359] Metadata directory not provided
I20260809 15:36:05.432696 10228 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 15:36:05.432940 10228 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 15:36:05.434950 10228 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786289732823074-3431-0/minicluster-data/ts-1/data/instance:
uuid: "7f9e63d41b3e46d9a25fb76d9463f92c"
format_stamp: "Formatted at 2026-08-09 15:36:05 on dist-test-slave-2w3w"
server_key: "63019c5a0e181d6d87f37c46da0da366"
server_key_iv: "307ca22b513284669e73afc9fe660ff5"
server_key_version: "encryptionkey@0"
I20260809 15:36:05.435650 10228 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786289732823074-3431-0/minicluster-data/ts-1/wal/instance:
uuid: "7f9e63d41b3e46d9a25fb76d9463f92c"
format_stamp: "Formatted at 2026-08-09 15:36:05 on dist-test-slave-2w3w"
server_key: "63019c5a0e181d6d87f37c46da0da366"
server_key_iv: "307ca22b513284669e73afc9fe660ff5"
server_key_version: "encryptionkey@0"
I20260809 15:36:05.439333 10228 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.003s sys 0.001s
I20260809 15:36:05.441720 10246 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 15:36:05.443022 10228 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260809 15:36:05.443159 10228 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786289732823074-3431-0/minicluster-data/ts-1/data,/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786289732823074-3431-0/minicluster-data/ts-1/wal
uuid: "7f9e63d41b3e46d9a25fb76d9463f92c"
format_stamp: "Formatted at 2026-08-09 15:36:05 on dist-test-slave-2w3w"
server_key: "63019c5a0e181d6d87f37c46da0da366"
server_key_iv: "307ca22b513284669e73afc9fe660ff5"
server_key_version: "encryptionkey@0"
I20260809 15:36:05.443272 10228 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786289732823074-3431-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786289732823074-3431-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786289732823074-3431-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 15:36:05.459458 10228 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 15:36:05.462388 10228 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 15:36:05.462603 10228 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 15:36:05.463220 10228 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 15:36:05.464046 10228 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 15:36:05.464119 10228 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 15:36:05.464190 10228 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 15:36:05.464241 10228 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 15:36:05.473356 10228 rpc_server.cc:307] RPC server started. Bound to: 127.3.89.194:36959
I20260809 15:36:05.473389 10366 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.3.89.194:36959 every 8 connection(s)
I20260809 15:36:05.474255 10228 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786289732823074-3431-0/minicluster-data/ts-1/data/info.pb
I20260809 15:36:05.480588 3431 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskm314YR/build/debug/bin/kudu as pid 10228
I20260809 15:36:05.480736 3431 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786289732823074-3431-0/minicluster-data/ts-1/wal/instance
I20260809 15:36:05.481041 3431 external_mini_cluster.cc:1428] Setting key 492bb67024323747add9566cf027894c
Aug 09 15:36:05 dist-test-slave-2w3w krb5kdc[9964](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786289765, etypes {rep=17 tkt=17 ses=17}, oryx/127.3.89.194@KRBTEST.COM for oryx/127.3.89.254@KRBTEST.COM
I20260809 15:36:05.489015 10057 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:36:05.476238 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.89.194:52773 (local address 127.3.89.254:46611)
0809 15:36:05.476402 (+ 164us) server_negotiation.cc:206] Beginning negotiation
0809 15:36:05.476406 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 15:36:05.477196 (+ 790us) server_negotiation.cc:407] Connection header received
0809 15:36:05.478403 (+ 1207us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 15:36:05.478407 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 15:36:05.478454 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 15:36:05.478535 (+ 81us) server_negotiation.cc:226] Negotiated authn=SASL
0809 15:36:05.479755 (+ 1220us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:36:05.480396 (+ 641us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:36:05.481673 (+ 1277us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:36:05.481945 (+ 272us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:36:05.482030 (+ 85us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:36:05.484875 (+ 2845us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 15:36:05.484879 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 15:36:05.484881 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 15:36:05.484905 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0809 15:36:05.486365 (+ 1460us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 15:36:05.486982 (+ 617us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 15:36:05.486989 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 15:36:05.486994 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0809 15:36:05.487096 (+ 102us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 15:36:05.487495 (+ 399us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 15:36:05.487500 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 15:36:05.487502 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 15:36:05.487754 (+ 252us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 15:36:05.487879 (+ 125us) server_negotiation.cc:1035] Waiting for connection context
0809 15:36:05.488617 (+ 738us) server_negotiation.cc:299] Negotiation successful
0809 15:36:05.488770 (+ 153us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":62}
I20260809 15:36:05.489660 10369 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:36:05.476606 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.89.254:46611 (local address 127.3.89.194:52773)
0809 15:36:05.477041 (+ 435us) negotiation.cc:107] Waiting for socket to connect
0809 15:36:05.477083 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0809 15:36:05.478200 (+ 1117us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 15:36:05.478694 (+ 494us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 15:36:05.478700 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 15:36:05.479133 (+ 433us) client_negotiation.cc:189] Negotiated authn=SASL
0809 15:36:05.479617 (+ 484us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:36:05.479630 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:36:05.480532 (+ 902us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:36:05.480535 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:36:05.481565 (+ 1030us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:36:05.481574 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:36:05.482097 (+ 523us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:36:05.482101 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:36:05.482303 (+ 202us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:36:05.482930 (+ 627us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 15:36:05.482946 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0809 15:36:05.484685 (+ 1739us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 15:36:05.486497 (+ 1812us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 15:36:05.486503 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 15:36:05.486518 (+ 15us) client_negotiation.cc:760] calling sasl_client_step()
0809 15:36:05.486791 (+ 273us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 15:36:05.487213 (+ 422us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 15:36:05.487217 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 15:36:05.487222 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0809 15:36:05.487388 (+ 166us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 15:36:05.487888 (+ 500us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 15:36:05.487893 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 15:36:05.488309 (+ 416us) client_negotiation.cc:769] Sending connection context
0809 15:36:05.488528 (+ 219us) client_negotiation.cc:240] Negotiation successful
0809 15:36:05.488790 (+ 262us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":259,"thread_start_us":108,"threads_started":1}
I20260809 15:36:05.490883 10367 heartbeater.cc:347] Connected to a master server at 127.3.89.254:46611
I20260809 15:36:05.491181 10367 heartbeater.cc:464] Registering TS with master...
I20260809 15:36:05.491786 10367 heartbeater.cc:511] Master 127.3.89.254:46611 requested a full tablet report, sending...
WARNING: no policy specified for oryx/127.3.89.195@KRBTEST.COM; defaulting to no policy
I20260809 15:36:05.493074 10014 ts_manager.cc:194] Registered new tserver with Master: 7f9e63d41b3e46d9a25fb76d9463f92c (127.3.89.194:36959)
I20260809 15:36:05.493682 10014 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.3.89.194@KRBTEST.COM'} at 127.3.89.194:52773
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.3.89.195@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.3.89.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786289732823074-3431-0/krb5kdc/oryx_127.3.89.195.keytab.
Entry for principal oryx/127.3.89.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786289732823074-3431-0/krb5kdc/oryx_127.3.89.195.keytab.
I20260809 15:36:05.510684 3431 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskm314YR/build/debug/bin/kudu
/tmp/dist-test-taskm314YR/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786289732823074-3431-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786289732823074-3431-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786289732823074-3431-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786289732823074-3431-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.3.89.195:0
--local_ip_for_outbound_sockets=127.3.89.195
--webserver_interface=127.3.89.195
--webserver_port=0
--tserver_master_addrs=127.3.89.254:46611
--keytab_file=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786289732823074-3431-0/krb5kdc/oryx_127.3.89.195.keytab
--principal=oryx/127.3.89.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.3.89.212:43259
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786289732823074-3431-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786289732823074-3431-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786289732823074-3431-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 15:36:05.621950 10373 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 15:36:05.622182 10373 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 15:36:05.622223 10373 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 15:36:05.626354 10373 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 15:36:05.626422 10373 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 15:36:05.626477 10373 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.3.89.195
I20260809 15:36:05.631661 10373 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.3.89.212:43259
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786289732823074-3431-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786289732823074-3431-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786289732823074-3431-0/krb5kdc/oryx_127.3.89.195.keytab
--principal=oryx/127.3.89.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.3.89.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786289732823074-3431-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.3.89.195
--webserver_port=0
--tserver_master_addrs=127.3.89.254:46611
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.10373
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.3.89.195
--log_dir=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786289732823074-3431-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 09 Aug 2026 14:46:55 UTC on ccycloud.quasar-yswkmf.root.comops.site
build id 4576540
I20260809 15:36:05.632644 10373 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 15:36:05.633589 10373 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 15:36:05.643057 10378 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 15:36:05.643087 10379 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 15:36:05.643245 10373 server_base.cc:1034] running on GCE node
W20260809 15:36:05.643057 10382 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 15:36:05.643869 10373 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 15:36:05.644461 10373 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 15:36:05.645627 10373 hybrid_clock.cc:648] HybridClock initialized: now 1786289765645607 us; error 33 us; skew 500 ppm
Aug 09 15:36:05 dist-test-slave-2w3w krb5kdc[9964](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786289765, etypes {rep=17 tkt=17 ses=17}, oryx/127.3.89.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 15:36:05.648638 10373 init.cc:377] Logged in from keytab as oryx/127.3.89.195@KRBTEST.COM (short username oryx)
I20260809 15:36:05.649462 10373 webserver.cc:460] Webserver started at http://127.3.89.195:35055/ using document root <none> and password file <none>
I20260809 15:36:05.650038 10373 fs_manager.cc:359] Metadata directory not provided
I20260809 15:36:05.650131 10373 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 15:36:05.650358 10373 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 15:36:05.652408 10373 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786289732823074-3431-0/minicluster-data/ts-2/data/instance:
uuid: "4d0727bc39574ad3b523c62e60bb1615"
format_stamp: "Formatted at 2026-08-09 15:36:05 on dist-test-slave-2w3w"
server_key: "4902f1fe9bbb1ab995c4f5920a6e18e2"
server_key_iv: "d7de3438af7bb5a35662fcd865f4b7ae"
server_key_version: "encryptionkey@0"
I20260809 15:36:05.652983 10373 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786289732823074-3431-0/minicluster-data/ts-2/wal/instance:
uuid: "4d0727bc39574ad3b523c62e60bb1615"
format_stamp: "Formatted at 2026-08-09 15:36:05 on dist-test-slave-2w3w"
server_key: "4902f1fe9bbb1ab995c4f5920a6e18e2"
server_key_iv: "d7de3438af7bb5a35662fcd865f4b7ae"
server_key_version: "encryptionkey@0"
I20260809 15:36:05.656596 10373 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.005s sys 0.000s
I20260809 15:36:05.658919 10390 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 15:36:05.660146 10373 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260809 15:36:05.660265 10373 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786289732823074-3431-0/minicluster-data/ts-2/data,/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786289732823074-3431-0/minicluster-data/ts-2/wal
uuid: "4d0727bc39574ad3b523c62e60bb1615"
format_stamp: "Formatted at 2026-08-09 15:36:05 on dist-test-slave-2w3w"
server_key: "4902f1fe9bbb1ab995c4f5920a6e18e2"
server_key_iv: "d7de3438af7bb5a35662fcd865f4b7ae"
server_key_version: "encryptionkey@0"
I20260809 15:36:05.660372 10373 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786289732823074-3431-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786289732823074-3431-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786289732823074-3431-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 15:36:05.676822 10373 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 15:36:05.680160 10373 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 15:36:05.680377 10373 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 15:36:05.680972 10373 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 15:36:05.681790 10373 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 15:36:05.681840 10373 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 15:36:05.681908 10373 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 15:36:05.681942 10373 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 15:36:05.690905 10373 rpc_server.cc:307] RPC server started. Bound to: 127.3.89.195:36983
I20260809 15:36:05.690933 10515 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.3.89.195:36983 every 8 connection(s)
I20260809 15:36:05.691795 10373 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786289732823074-3431-0/minicluster-data/ts-2/data/info.pb
I20260809 15:36:05.697829 3431 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskm314YR/build/debug/bin/kudu as pid 10373
I20260809 15:36:05.698009 3431 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786289732823074-3431-0/minicluster-data/ts-2/wal/instance
I20260809 15:36:05.698299 3431 external_mini_cluster.cc:1428] Setting key 6328dbd4b1913093bfeedfb8204432c8
Aug 09 15:36:05 dist-test-slave-2w3w krb5kdc[9964](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786289765, etypes {rep=17 tkt=17 ses=17}, oryx/127.3.89.195@KRBTEST.COM for oryx/127.3.89.254@KRBTEST.COM
I20260809 15:36:05.705564 10057 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:36:05.693517 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.89.195:49951 (local address 127.3.89.254:46611)
0809 15:36:05.693666 (+ 149us) server_negotiation.cc:206] Beginning negotiation
0809 15:36:05.693669 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0809 15:36:05.694469 (+ 800us) server_negotiation.cc:407] Connection header received
0809 15:36:05.695794 (+ 1325us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 15:36:05.695798 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 15:36:05.695856 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 15:36:05.695966 (+ 110us) server_negotiation.cc:226] Negotiated authn=SASL
0809 15:36:05.697078 (+ 1112us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:36:05.697716 (+ 638us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:36:05.699276 (+ 1560us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:36:05.699623 (+ 347us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:36:05.699739 (+ 116us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:36:05.702379 (+ 2640us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 15:36:05.702382 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 15:36:05.702384 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 15:36:05.702408 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0809 15:36:05.703761 (+ 1353us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 15:36:05.704246 (+ 485us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 15:36:05.704249 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 15:36:05.704250 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 15:36:05.704329 (+ 79us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 15:36:05.704622 (+ 293us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 15:36:05.704624 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 15:36:05.704625 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 15:36:05.704800 (+ 175us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 15:36:05.704899 (+ 99us) server_negotiation.cc:1035] Waiting for connection context
0809 15:36:05.705349 (+ 450us) server_negotiation.cc:299] Negotiation successful
0809 15:36:05.705444 (+ 95us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":60}
I20260809 15:36:05.706454 10519 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:36:05.693885 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.89.254:46611 (local address 127.3.89.195:49951)
0809 15:36:05.694322 (+ 437us) negotiation.cc:107] Waiting for socket to connect
0809 15:36:05.694363 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0809 15:36:05.695568 (+ 1205us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 15:36:05.696128 (+ 560us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 15:36:05.696135 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 15:36:05.696483 (+ 348us) client_negotiation.cc:189] Negotiated authn=SASL
0809 15:36:05.696936 (+ 453us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:36:05.696949 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:36:05.697865 (+ 916us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:36:05.697868 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:36:05.699095 (+ 1227us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:36:05.699108 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:36:05.699798 (+ 690us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:36:05.699802 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:36:05.700005 (+ 203us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:36:05.700709 (+ 704us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 15:36:05.700725 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0809 15:36:05.702202 (+ 1477us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 15:36:05.703879 (+ 1677us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 15:36:05.703883 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 15:36:05.703895 (+ 12us) client_negotiation.cc:760] calling sasl_client_step()
0809 15:36:05.704155 (+ 260us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 15:36:05.704409 (+ 254us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 15:36:05.704412 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 15:36:05.704413 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0809 15:36:05.704542 (+ 129us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 15:36:05.704908 (+ 366us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 15:36:05.704913 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 15:36:05.705147 (+ 234us) client_negotiation.cc:769] Sending connection context
0809 15:36:05.705303 (+ 156us) client_negotiation.cc:240] Negotiation successful
0809 15:36:05.705588 (+ 285us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":270,"thread_start_us":108,"threads_started":1}
I20260809 15:36:05.707647 10517 heartbeater.cc:347] Connected to a master server at 127.3.89.254:46611
I20260809 15:36:05.707844 10517 heartbeater.cc:464] Registering TS with master...
I20260809 15:36:05.708267 10517 heartbeater.cc:511] Master 127.3.89.254:46611 requested a full tablet report, sending...
I20260809 15:36:05.709229 10014 ts_manager.cc:194] Registered new tserver with Master: 4d0727bc39574ad3b523c62e60bb1615 (127.3.89.195:36983)
I20260809 15:36:05.709797 10014 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.3.89.195@KRBTEST.COM'} at 127.3.89.195:49951
I20260809 15:36:05.712167 3431 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
WARNING: no policy specified for oryx/127.3.89.253@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.3.89.253@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.3.89.253 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786289732823074-3431-0/krb5kdc/oryx_127.3.89.253.keytab.
Entry for principal oryx/127.3.89.253 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786289732823074-3431-0/krb5kdc/oryx_127.3.89.253.keytab.
I20260809 15:36:05.742166 3431 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskm314YR/build/debug/bin/kudu
/tmp/dist-test-taskm314YR/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786289732823074-3431-0/minicluster-data/master-1/wal
--fs_data_dirs=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786289732823074-3431-0/minicluster-data/master-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786289732823074-3431-0/minicluster-data/master-1/logs
--server_dump_info_path=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786289732823074-3431-0/minicluster-data/master-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.3.89.253:42075
--webserver_interface=127.3.89.253
--webserver_port=0
--keytab_file=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786289732823074-3431-0/krb5kdc/oryx_127.3.89.253.keytab
--principal=oryx/127.3.89.253
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.3.89.212:43259
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.3.89.254:46611,127.3.89.253:42075
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786289732823074-3431-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786289732823074-3431-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786289732823074-3431-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 15:36:05.857674 10522 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 15:36:05.857923 10522 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 15:36:05.857962 10522 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 15:36:05.861960 10522 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 15:36:05.862020 10522 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 15:36:05.862038 10522 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 15:36:05.862056 10522 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 15:36:05.862071 10522 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 15:36:05.867384 10522 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.3.89.212:43259
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786289732823074-3431-0/minicluster-data/master-1/data
--fs_wal_dir=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786289732823074-3431-0/minicluster-data/master-1/wal
--ipki_ca_key_size=768
--master_addresses=127.3.89.254:46611,127.3.89.253:42075
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786289732823074-3431-0/krb5kdc/oryx_127.3.89.253.keytab
--principal=oryx/127.3.89.253
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.3.89.253:42075
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786289732823074-3431-0/minicluster-data/master-1/data/info.pb
--webserver_interface=127.3.89.253
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.10522
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786289732823074-3431-0/minicluster-data/master-1/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 09 Aug 2026 14:46:55 UTC on ccycloud.quasar-yswkmf.root.comops.site
build id 4576540
W20260809 15:36:05.867597 10522 master_options.cc:55] Only 2 masters are specified by master_addresses_flag ('127.3.89.254:46611,127.3.89.253:42075'), but minimum of 3 are required to tolerate failures of any one master. It is recommended to use at least 3 masters.
I20260809 15:36:05.868436 10522 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 15:36:05.869441 10522 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 15:36:05.877871 10528 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 15:36:05.878036 10522 server_base.cc:1034] running on GCE node
W20260809 15:36:05.877919 10530 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 15:36:05.877871 10527 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 15:36:05.878671 10522 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 15:36:05.879899 10522 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 15:36:05.881093 10522 hybrid_clock.cc:648] HybridClock initialized: now 1786289765881077 us; error 27 us; skew 500 ppm
Aug 09 15:36:05 dist-test-slave-2w3w krb5kdc[9964](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786289765, etypes {rep=17 tkt=17 ses=17}, oryx/127.3.89.253@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 15:36:05.883960 10522 init.cc:377] Logged in from keytab as oryx/127.3.89.253@KRBTEST.COM (short username oryx)
I20260809 15:36:05.884953 10522 webserver.cc:460] Webserver started at http://127.3.89.253:45853/ using document root <none> and password file <none>
I20260809 15:36:05.885591 10522 fs_manager.cc:359] Metadata directory not provided
I20260809 15:36:05.885691 10522 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 15:36:05.885952 10522 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 15:36:05.888401 10522 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786289732823074-3431-0/minicluster-data/master-1/data/instance:
uuid: "d36c8ca8de404275bc63795e17cb197e"
format_stamp: "Formatted at 2026-08-09 15:36:05 on dist-test-slave-2w3w"
server_key: "35be02d71884d25fb755ae017044143b"
server_key_iv: "98ba96cce0359522d9c7551d13cfb1e1"
server_key_version: "encryptionkey@0"
I20260809 15:36:05.889004 10522 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786289732823074-3431-0/minicluster-data/master-1/wal/instance:
uuid: "d36c8ca8de404275bc63795e17cb197e"
format_stamp: "Formatted at 2026-08-09 15:36:05 on dist-test-slave-2w3w"
server_key: "35be02d71884d25fb755ae017044143b"
server_key_iv: "98ba96cce0359522d9c7551d13cfb1e1"
server_key_version: "encryptionkey@0"
I20260809 15:36:05.893046 10522 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260809 15:36:05.895799 10539 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 15:36:05.897356 10522 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.001s
I20260809 15:36:05.897523 10522 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786289732823074-3431-0/minicluster-data/master-1/data,/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786289732823074-3431-0/minicluster-data/master-1/wal
uuid: "d36c8ca8de404275bc63795e17cb197e"
format_stamp: "Formatted at 2026-08-09 15:36:05 on dist-test-slave-2w3w"
server_key: "35be02d71884d25fb755ae017044143b"
server_key_iv: "98ba96cce0359522d9c7551d13cfb1e1"
server_key_version: "encryptionkey@0"
I20260809 15:36:05.897655 10522 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786289732823074-3431-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786289732823074-3431-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786289732823074-3431-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 15:36:05.911806 10522 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 15:36:05.915237 10522 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 15:36:05.915469 10522 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 15:36:05.923264 10522 rpc_server.cc:307] RPC server started. Bound to: 127.3.89.253:42075
I20260809 15:36:05.923584 10596 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.3.89.253:42075 every 8 connection(s)
I20260809 15:36:05.924614 10522 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786289732823074-3431-0/minicluster-data/master-1/data/info.pb
I20260809 15:36:05.927733 10597 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 15:36:05.930768 3431 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskm314YR/build/debug/bin/kudu as pid 10522
I20260809 15:36:05.931728 10597 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.3.89.254" port: 46611 } has no permanent_uuid. Determining permanent_uuid...
Aug 09 15:36:05 dist-test-slave-2w3w krb5kdc[9964](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786289765, etypes {rep=17 tkt=17 ses=17}, oryx/127.3.89.253@KRBTEST.COM for oryx/127.3.89.254@KRBTEST.COM
Aug 09 15:36:05 dist-test-slave-2w3w krb5kdc[9964](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786289764, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.3.89.253@KRBTEST.COM
I20260809 15:36:05.944810 10057 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:36:05.933656 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53130 (local address 127.3.89.254:46611)
0809 15:36:05.933779 (+ 123us) server_negotiation.cc:206] Beginning negotiation
0809 15:36:05.933797 (+ 18us) server_negotiation.cc:399] Waiting for connection header
0809 15:36:05.934128 (+ 331us) server_negotiation.cc:407] Connection header received
0809 15:36:05.934432 (+ 304us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 15:36:05.934435 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 15:36:05.934490 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 15:36:05.934590 (+ 100us) server_negotiation.cc:226] Negotiated authn=SASL
0809 15:36:05.935489 (+ 899us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:36:05.936040 (+ 551us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:36:05.937204 (+ 1164us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:36:05.937426 (+ 222us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:36:05.937538 (+ 112us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:36:05.940517 (+ 2979us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 15:36:05.940524 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 15:36:05.940528 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 15:36:05.940558 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0809 15:36:05.942280 (+ 1722us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 15:36:05.942907 (+ 627us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 15:36:05.942911 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 15:36:05.942913 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 15:36:05.943036 (+ 123us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 15:36:05.943396 (+ 360us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 15:36:05.943402 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 15:36:05.943404 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 15:36:05.943694 (+ 290us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 15:36:05.943771 (+ 77us) server_negotiation.cc:1035] Waiting for connection context
0809 15:36:05.944482 (+ 711us) server_negotiation.cc:299] Negotiation successful
0809 15:36:05.944610 (+ 128us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":47}
I20260809 15:36:05.945699 10600 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:36:05.932418 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36440 (local address 127.3.89.253:42075)
0809 15:36:05.932898 (+ 480us) server_negotiation.cc:206] Beginning negotiation
0809 15:36:05.932913 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0809 15:36:05.932952 (+ 39us) server_negotiation.cc:407] Connection header received
0809 15:36:05.933881 (+ 929us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 15:36:05.933905 (+ 24us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 15:36:05.934275 (+ 370us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 15:36:05.934517 (+ 242us) server_negotiation.cc:226] Negotiated authn=SASL
0809 15:36:05.935544 (+ 1027us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:36:05.936593 (+ 1049us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:36:05.937504 (+ 911us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:36:05.937903 (+ 399us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:36:05.938060 (+ 157us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:36:05.941448 (+ 3388us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 15:36:05.941453 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 15:36:05.941494 (+ 41us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 15:36:05.941522 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0809 15:36:05.943443 (+ 1921us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 15:36:05.943940 (+ 497us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 15:36:05.943945 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 15:36:05.943946 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 15:36:05.944019 (+ 73us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 15:36:05.944337 (+ 318us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 15:36:05.944339 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 15:36:05.944340 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 15:36:05.944664 (+ 324us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 15:36:05.944866 (+ 202us) server_negotiation.cc:1035] Waiting for connection context
0809 15:36:05.945112 (+ 246us) server_negotiation.cc:299] Negotiation successful
0809 15:36:05.945263 (+ 151us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":316,"thread_start_us":132,"threads_started":1}
I20260809 15:36:05.945706 10601 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:36:05.933617 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.89.254:46611 (local address 127.0.0.1:53130)
0809 15:36:05.933995 (+ 378us) negotiation.cc:107] Waiting for socket to connect
0809 15:36:05.934034 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0809 15:36:05.934258 (+ 224us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 15:36:05.934640 (+ 382us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 15:36:05.934645 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 15:36:05.935066 (+ 421us) client_negotiation.cc:189] Negotiated authn=SASL
0809 15:36:05.935326 (+ 260us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:36:05.935339 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:36:05.936185 (+ 846us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:36:05.936189 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:36:05.937071 (+ 882us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:36:05.937081 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:36:05.937571 (+ 490us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:36:05.937575 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:36:05.937747 (+ 172us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:36:05.938384 (+ 637us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 15:36:05.938400 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0809 15:36:05.940357 (+ 1957us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 15:36:05.942410 (+ 2053us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 15:36:05.942422 (+ 12us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 15:36:05.942435 (+ 13us) client_negotiation.cc:760] calling sasl_client_step()
0809 15:36:05.942800 (+ 365us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 15:36:05.943139 (+ 339us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 15:36:05.943141 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 15:36:05.943143 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 15:36:05.943292 (+ 149us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 15:36:05.943846 (+ 554us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 15:36:05.943857 (+ 11us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 15:36:05.944191 (+ 334us) client_negotiation.cc:769] Sending connection context
0809 15:36:05.944413 (+ 222us) client_negotiation.cc:240] Negotiation successful
0809 15:36:05.944681 (+ 268us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":245,"thread_start_us":150,"threads_started":1}
I20260809 15:36:05.947327 10597 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.3.89.253" port: 42075 } has no permanent_uuid. Determining permanent_uuid...
Aug 09 15:36:05 dist-test-slave-2w3w krb5kdc[9964](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786289765, etypes {rep=17 tkt=17 ses=17}, oryx/127.3.89.253@KRBTEST.COM for oryx/127.3.89.253@KRBTEST.COM
I20260809 15:36:05.955405 10601 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:36:05.947860 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.89.253:42075 (local address 127.0.0.1:36446)
0809 15:36:05.948000 (+ 140us) negotiation.cc:107] Waiting for socket to connect
0809 15:36:05.948018 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0809 15:36:05.948100 (+ 82us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 15:36:05.948330 (+ 230us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 15:36:05.948332 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 15:36:05.948458 (+ 126us) client_negotiation.cc:189] Negotiated authn=SASL
0809 15:36:05.948543 (+ 85us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:36:05.948548 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:36:05.949237 (+ 689us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:36:05.949239 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:36:05.949724 (+ 485us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:36:05.949730 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:36:05.950743 (+ 1013us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:36:05.950746 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:36:05.950817 (+ 71us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:36:05.951244 (+ 427us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 15:36:05.951255 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0809 15:36:05.952378 (+ 1123us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 15:36:05.954116 (+ 1738us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 15:36:05.954119 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 15:36:05.954120 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0809 15:36:05.954292 (+ 172us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 15:36:05.954531 (+ 239us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 15:36:05.954533 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 15:36:05.954534 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0809 15:36:05.954602 (+ 68us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 15:36:05.954931 (+ 329us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 15:36:05.954933 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 15:36:05.955064 (+ 131us) client_negotiation.cc:769] Sending connection context
0809 15:36:05.955149 (+ 85us) client_negotiation.cc:240] Negotiation successful
0809 15:36:05.955302 (+ 153us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":46}
I20260809 15:36:05.955612 10600 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:36:05.947855 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36446 (local address 127.3.89.253:42075)
0809 15:36:05.947984 (+ 129us) server_negotiation.cc:206] Beginning negotiation
0809 15:36:05.947988 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 15:36:05.948061 (+ 73us) server_negotiation.cc:407] Connection header received
0809 15:36:05.948202 (+ 141us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 15:36:05.948205 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 15:36:05.948248 (+ 43us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 15:36:05.948312 (+ 64us) server_negotiation.cc:226] Negotiated authn=SASL
0809 15:36:05.948626 (+ 314us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:36:05.949148 (+ 522us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:36:05.949824 (+ 676us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:36:05.950642 (+ 818us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:36:05.950731 (+ 89us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:36:05.952500 (+ 1769us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 15:36:05.952503 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 15:36:05.952505 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 15:36:05.952526 (+ 21us) server_negotiation.cc:922] calling sasl_server_start()
0809 15:36:05.954024 (+ 1498us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 15:36:05.954376 (+ 352us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 15:36:05.954378 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 15:36:05.954380 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 15:36:05.954452 (+ 72us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 15:36:05.954678 (+ 226us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 15:36:05.954680 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 15:36:05.954682 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 15:36:05.954849 (+ 167us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 15:36:05.954930 (+ 81us) server_negotiation.cc:1035] Waiting for connection context
0809 15:36:05.955245 (+ 315us) server_negotiation.cc:299] Negotiation successful
0809 15:36:05.955405 (+ 160us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":42}
I20260809 15:36:05.958206 10597 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d36c8ca8de404275bc63795e17cb197e: Bootstrap starting.
I20260809 15:36:05.960451 10597 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P d36c8ca8de404275bc63795e17cb197e: Neither blocks nor log segments found. Creating new log.
I20260809 15:36:05.961167 10597 log.cc:824] T 00000000000000000000000000000000 P d36c8ca8de404275bc63795e17cb197e: Log is configured to *not* fsync() on all Append() calls
I20260809 15:36:05.963043 10597 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d36c8ca8de404275bc63795e17cb197e: No bootstrap required, opened a new log
I20260809 15:36:05.966152 10597 raft_consensus.cc:348] T 00000000000000000000000000000000 P d36c8ca8de404275bc63795e17cb197e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b7ef4cef863642f19bd9f01ae2792fe3" member_type: VOTER last_known_addr { host: "127.3.89.254" port: 46611 } } peers { permanent_uuid: "d36c8ca8de404275bc63795e17cb197e" member_type: VOTER last_known_addr { host: "127.3.89.253" port: 42075 } }
I20260809 15:36:05.966378 10597 raft_consensus.cc:374] T 00000000000000000000000000000000 P d36c8ca8de404275bc63795e17cb197e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 15:36:05.966461 10597 raft_consensus.cc:729] T 00000000000000000000000000000000 P d36c8ca8de404275bc63795e17cb197e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d36c8ca8de404275bc63795e17cb197e, State: Initialized, Role: FOLLOWER
I20260809 15:36:05.966928 10597 consensus_queue.cc:260] T 00000000000000000000000000000000 P d36c8ca8de404275bc63795e17cb197e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b7ef4cef863642f19bd9f01ae2792fe3" member_type: VOTER last_known_addr { host: "127.3.89.254" port: 46611 } } peers { permanent_uuid: "d36c8ca8de404275bc63795e17cb197e" member_type: VOTER last_known_addr { host: "127.3.89.253" port: 42075 } }
I20260809 15:36:05.967468 10603 sys_catalog.cc:455] T 00000000000000000000000000000000 P d36c8ca8de404275bc63795e17cb197e [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b7ef4cef863642f19bd9f01ae2792fe3" member_type: VOTER last_known_addr { host: "127.3.89.254" port: 46611 } } peers { permanent_uuid: "d36c8ca8de404275bc63795e17cb197e" member_type: VOTER last_known_addr { host: "127.3.89.253" port: 42075 } } }
I20260809 15:36:05.967592 10603 sys_catalog.cc:458] T 00000000000000000000000000000000 P d36c8ca8de404275bc63795e17cb197e [sys.catalog]: This master's current role is: FOLLOWER
I20260809 15:36:05.968209 10597 sys_catalog.cc:564] T 00000000000000000000000000000000 P d36c8ca8de404275bc63795e17cb197e [sys.catalog]: configured and running, proceeding with master startup.
W20260809 15:36:05.973011 10615 catalog_manager.cc:1558] T 00000000000000000000000000000000 P d36c8ca8de404275bc63795e17cb197e: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260809 15:36:05.973106 10615 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260809 15:36:05.975358 10522 master_runner.cc:428] Detected that this master d36c8ca8de404275bc63795e17cb197e is joining an existing cluster
I20260809 15:36:05.975414 10522 master_runner.cc:432] Initiating AddMaster RPC to add 127.3.89.253:42075
I20260809 15:36:05.986049 10625 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:36:05.977821 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.89.254:46611 (local address 127.0.0.1:53142)
0809 15:36:05.978625 (+ 804us) negotiation.cc:107] Waiting for socket to connect
0809 15:36:05.978641 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0809 15:36:05.978783 (+ 142us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 15:36:05.979084 (+ 301us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 15:36:05.979087 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 15:36:05.979330 (+ 243us) client_negotiation.cc:189] Negotiated authn=SASL
0809 15:36:05.979429 (+ 99us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:36:05.979435 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:36:05.980196 (+ 761us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:36:05.980202 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:36:05.980832 (+ 630us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:36:05.980840 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:36:05.981311 (+ 471us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:36:05.981315 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:36:05.981452 (+ 137us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:36:05.982005 (+ 553us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 15:36:05.982020 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0809 15:36:05.982495 (+ 475us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 15:36:05.984270 (+ 1775us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 15:36:05.984275 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 15:36:05.984277 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 15:36:05.984550 (+ 273us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 15:36:05.984847 (+ 297us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 15:36:05.984855 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 15:36:05.984858 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0809 15:36:05.984956 (+ 98us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 15:36:05.985433 (+ 477us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 15:36:05.985436 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 15:36:05.985514 (+ 78us) client_negotiation.cc:769] Sending connection context
0809 15:36:05.985667 (+ 153us) client_negotiation.cc:240] Negotiation successful
0809 15:36:05.985814 (+ 147us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":722,"mutex_wait_us":28,"thread_start_us":95,"threads_started":1}
I20260809 15:36:05.986099 10057 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:36:05.977824 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53142 (local address 127.3.89.254:46611)
0809 15:36:05.977954 (+ 130us) server_negotiation.cc:206] Beginning negotiation
0809 15:36:05.977967 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0809 15:36:05.978711 (+ 744us) server_negotiation.cc:407] Connection header received
0809 15:36:05.978903 (+ 192us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 15:36:05.978907 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 15:36:05.978987 (+ 80us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 15:36:05.979071 (+ 84us) server_negotiation.cc:226] Negotiated authn=SASL
0809 15:36:05.979548 (+ 477us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:36:05.980085 (+ 537us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:36:05.980952 (+ 867us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:36:05.981181 (+ 229us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:36:05.981300 (+ 119us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:36:05.982628 (+ 1328us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 15:36:05.982632 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 15:36:05.982634 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 15:36:05.982661 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0809 15:36:05.984149 (+ 1488us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 15:36:05.984642 (+ 493us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 15:36:05.984647 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 15:36:05.984649 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 15:36:05.984742 (+ 93us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 15:36:05.985087 (+ 345us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 15:36:05.985091 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 15:36:05.985093 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 15:36:05.985329 (+ 236us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 15:36:05.985439 (+ 110us) server_negotiation.cc:1035] Waiting for connection context
0809 15:36:05.985803 (+ 364us) server_negotiation.cc:299] Negotiation successful
0809 15:36:05.985918 (+ 115us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":50}
I20260809 15:36:05.987942 10600 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:36:05.978307 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36458 (local address 127.3.89.253:42075)
0809 15:36:05.978401 (+ 94us) server_negotiation.cc:206] Beginning negotiation
0809 15:36:05.978405 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 15:36:05.978787 (+ 382us) server_negotiation.cc:407] Connection header received
0809 15:36:05.978937 (+ 150us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 15:36:05.978940 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 15:36:05.979010 (+ 70us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 15:36:05.979086 (+ 76us) server_negotiation.cc:226] Negotiated authn=SASL
0809 15:36:05.979662 (+ 576us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:36:05.980409 (+ 747us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:36:05.981963 (+ 1554us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:36:05.982288 (+ 325us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:36:05.982401 (+ 113us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:36:05.984084 (+ 1683us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 15:36:05.984089 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 15:36:05.984093 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 15:36:05.984117 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0809 15:36:05.986210 (+ 2093us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 15:36:05.986653 (+ 443us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 15:36:05.986657 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 15:36:05.986658 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 15:36:05.986729 (+ 71us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 15:36:05.987052 (+ 323us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 15:36:05.987055 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 15:36:05.987057 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 15:36:05.987271 (+ 214us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 15:36:05.987340 (+ 69us) server_negotiation.cc:1035] Waiting for connection context
0809 15:36:05.987688 (+ 348us) server_negotiation.cc:299] Negotiation successful
0809 15:36:05.987793 (+ 105us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":31}
I20260809 15:36:05.988473 10624 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:36:05.978244 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.89.253:42075 (local address 127.0.0.1:36458)
0809 15:36:05.978704 (+ 460us) negotiation.cc:107] Waiting for socket to connect
0809 15:36:05.978722 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0809 15:36:05.978829 (+ 107us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 15:36:05.979184 (+ 355us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 15:36:05.979187 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 15:36:05.979426 (+ 239us) client_negotiation.cc:189] Negotiated authn=SASL
0809 15:36:05.979547 (+ 121us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:36:05.979554 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:36:05.980543 (+ 989us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:36:05.980547 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:36:05.981504 (+ 957us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:36:05.981515 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:36:05.982422 (+ 907us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:36:05.982426 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:36:05.982524 (+ 98us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:36:05.983035 (+ 511us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 15:36:05.983050 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0809 15:36:05.983923 (+ 873us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 15:36:05.986330 (+ 2407us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 15:36:05.986333 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 15:36:05.986335 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 15:36:05.986554 (+ 219us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 15:36:05.986825 (+ 271us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 15:36:05.986827 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 15:36:05.986829 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 15:36:05.986904 (+ 75us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 15:36:05.987451 (+ 547us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 15:36:05.987454 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 15:36:05.987531 (+ 77us) client_negotiation.cc:769] Sending connection context
0809 15:36:05.988132 (+ 601us) client_negotiation.cc:240] Negotiation successful
0809 15:36:05.988262 (+ 130us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":380,"mutex_wait_us":26,"thread_start_us":194,"threads_started":1}
Aug 09 15:36:05 dist-test-slave-2w3w krb5kdc[9964](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786289764, etypes {rep=17 tkt=17 ses=17}, oryx/127.3.89.254@KRBTEST.COM for oryx/127.3.89.253@KRBTEST.COM
I20260809 15:36:06.000854 10600 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:36:05.991007 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36468 (local address 127.3.89.253:42075)
0809 15:36:05.991109 (+ 102us) server_negotiation.cc:206] Beginning negotiation
0809 15:36:05.991113 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 15:36:05.991421 (+ 308us) server_negotiation.cc:407] Connection header received
0809 15:36:05.991598 (+ 177us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 15:36:05.991601 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 15:36:05.991652 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 15:36:05.991738 (+ 86us) server_negotiation.cc:226] Negotiated authn=SASL
0809 15:36:05.992358 (+ 620us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:36:05.993113 (+ 755us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:36:05.994213 (+ 1100us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:36:05.994550 (+ 337us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:36:05.994653 (+ 103us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:36:05.997153 (+ 2500us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 15:36:05.997157 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 15:36:05.997160 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 15:36:05.997186 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0809 15:36:05.998738 (+ 1552us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 15:36:05.999340 (+ 602us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 15:36:05.999343 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 15:36:05.999345 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 15:36:05.999427 (+ 82us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 15:36:05.999764 (+ 337us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 15:36:05.999767 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 15:36:05.999769 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 15:36:05.999991 (+ 222us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 15:36:06.000088 (+ 97us) server_negotiation.cc:1035] Waiting for connection context
0809 15:36:06.000580 (+ 492us) server_negotiation.cc:299] Negotiation successful
0809 15:36:06.000696 (+ 116us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":31}
I20260809 15:36:06.001045 10626 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:36:05.990935 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.89.253:42075 (local address 127.0.0.1:36468)
0809 15:36:05.991285 (+ 350us) negotiation.cc:107] Waiting for socket to connect
0809 15:36:05.991338 (+ 53us) client_negotiation.cc:174] Beginning negotiation
0809 15:36:05.991464 (+ 126us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 15:36:05.991779 (+ 315us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 15:36:05.991785 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 15:36:05.992119 (+ 334us) client_negotiation.cc:189] Negotiated authn=SASL
0809 15:36:05.992246 (+ 127us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:36:05.992252 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:36:05.993252 (+ 1000us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:36:05.993255 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:36:05.994088 (+ 833us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:36:05.994098 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:36:05.994662 (+ 564us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:36:05.994665 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:36:05.994768 (+ 103us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:36:05.995384 (+ 616us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 15:36:05.995397 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0809 15:36:05.997015 (+ 1618us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 15:36:05.998859 (+ 1844us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 15:36:05.998870 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 15:36:05.998876 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0809 15:36:05.999229 (+ 353us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 15:36:05.999536 (+ 307us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 15:36:05.999540 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 15:36:05.999542 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 15:36:05.999649 (+ 107us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 15:36:06.000122 (+ 473us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 15:36:06.000135 (+ 13us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 15:36:06.000317 (+ 182us) client_negotiation.cc:769] Sending connection context
0809 15:36:06.000489 (+ 172us) client_negotiation.cc:240] Negotiation successful
0809 15:36:06.000741 (+ 252us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":176,"thread_start_us":99,"threads_started":1}
I20260809 15:36:06.003562 10014 catalog_manager.cc:6979] Initiating ChangeConfig request to add master 127.3.89.253:42075: tablet_id: "00000000000000000000000000000000"
type: ADD_PEER
server {
permanent_uuid: "d36c8ca8de404275bc63795e17cb197e"
member_type: NON_VOTER
last_known_addr {
host: "127.3.89.253"
port: 42075
}
attrs {
promote: true
}
}
dest_uuid: "b7ef4cef863642f19bd9f01ae2792fe3"
cas_config_opid_index: -1
I20260809 15:36:06.004359 10014 consensus_queue.cc:237] T 00000000000000000000000000000000 P b7ef4cef863642f19bd9f01ae2792fe3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "b7ef4cef863642f19bd9f01ae2792fe3" member_type: VOTER last_known_addr { host: "127.3.89.254" port: 46611 } } peers { permanent_uuid: "d36c8ca8de404275bc63795e17cb197e" member_type: NON_VOTER last_known_addr { host: "127.3.89.253" port: 42075 } attrs { promote: true } }
I20260809 15:36:06.006285 10628 raft_consensus.cc:2932] T 00000000000000000000000000000000 P b7ef4cef863642f19bd9f01ae2792fe3 [term 1 LEADER]: Committing config change with OpId 1.5: config changed from index -1 to 5, NON_VOTER d36c8ca8de404275bc63795e17cb197e (127.3.89.253) added. New config: { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "b7ef4cef863642f19bd9f01ae2792fe3" member_type: VOTER last_known_addr { host: "127.3.89.254" port: 46611 } } peers { permanent_uuid: "d36c8ca8de404275bc63795e17cb197e" member_type: NON_VOTER last_known_addr { host: "127.3.89.253" port: 42075 } attrs { promote: true } } }
I20260809 15:36:06.007261 10568 raft_consensus.cc:3037] T 00000000000000000000000000000000 P d36c8ca8de404275bc63795e17cb197e [term 0 FOLLOWER]: Advancing to term 1
I20260809 15:36:06.008769 10568 raft_consensus.cc:1261] T 00000000000000000000000000000000 P d36c8ca8de404275bc63795e17cb197e [term 1 FOLLOWER]: Refusing update from remote peer b7ef4cef863642f19bd9f01ae2792fe3: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20260809 15:36:06.009060 10628 catalog_manager.cc:6983] Successfully completed master ChangeConfig request to add master 127.3.89.253:42075
I20260809 15:36:06.009332 10603 raft_consensus.cc:482] T 00000000000000000000000000000000 P d36c8ca8de404275bc63795e17cb197e [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 15:36:06.009495 10603 raft_consensus.cc:504] T 00000000000000000000000000000000 P d36c8ca8de404275bc63795e17cb197e [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b7ef4cef863642f19bd9f01ae2792fe3" member_type: VOTER last_known_addr { host: "127.3.89.254" port: 46611 } } peers { permanent_uuid: "d36c8ca8de404275bc63795e17cb197e" member_type: VOTER last_known_addr { host: "127.3.89.253" port: 42075 } }
I20260809 15:36:06.009897 10522 master.cc:537] Master@127.3.89.253:42075 shutting down...
I20260809 15:36:06.009945 10628 consensus_queue.cc:1035] T 00000000000000000000000000000000 P b7ef4cef863642f19bd9f01ae2792fe3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d36c8ca8de404275bc63795e17cb197e" member_type: NON_VOTER last_known_addr { host: "127.3.89.253" port: 42075 } attrs { promote: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 15:36:06.010274 10603 leader_election.cc:288] T 00000000000000000000000000000000 P d36c8ca8de404275bc63795e17cb197e [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers b7ef4cef863642f19bd9f01ae2792fe3 (127.3.89.254:46611)
I20260809 15:36:06.010845 10026 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "d36c8ca8de404275bc63795e17cb197e" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b7ef4cef863642f19bd9f01ae2792fe3" is_pre_election: true
WARNING: no policy specified for oryx/127.3.89.252@KRBTEST.COM; defaulting to no policy
I20260809 15:36:06.011590 10629 sys_catalog.cc:455] T 00000000000000000000000000000000 P b7ef4cef863642f19bd9f01ae2792fe3 [sys.catalog]: SysCatalogTable state changed. Reason: Config change replication complete. Latest consensus state: current_term: 1 leader_uuid: "b7ef4cef863642f19bd9f01ae2792fe3" committed_config { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "b7ef4cef863642f19bd9f01ae2792fe3" member_type: VOTER last_known_addr { host: "127.3.89.254" port: 46611 } } peers { permanent_uuid: "d36c8ca8de404275bc63795e17cb197e" member_type: NON_VOTER last_known_addr { host: "127.3.89.253" port: 42075 } attrs { promote: true } } }
I20260809 15:36:06.011584 10543 leader_election.cc:302] T 00000000000000000000000000000000 P d36c8ca8de404275bc63795e17cb197e [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 2 responses out of 2 voters: 1 yes votes; 1 no votes. yes voters: d36c8ca8de404275bc63795e17cb197e; no voters: b7ef4cef863642f19bd9f01ae2792fe3
I20260809 15:36:06.011718 10629 sys_catalog.cc:458] T 00000000000000000000000000000000 P b7ef4cef863642f19bd9f01ae2792fe3 [sys.catalog]: This master's current role is: LEADER
I20260809 15:36:06.011955 10603 raft_consensus.cc:2726] T 00000000000000000000000000000000 P d36c8ca8de404275bc63795e17cb197e [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
W20260809 15:36:06.012861 10001 proxy.cc:239] Call had error, refreshing address and retrying: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on Master
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.3.89.252@KRBTEST.COM" created.
I20260809 15:36:06.015636 10522 raft_consensus.cc:2229] T 00000000000000000000000000000000 P d36c8ca8de404275bc63795e17cb197e [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 15:36:06.015918 10522 raft_consensus.cc:2258] T 00000000000000000000000000000000 P d36c8ca8de404275bc63795e17cb197e [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 15:36:06.016045 10522 tablet_replica.cc:330] T 00000000000000000000000000000000 P d36c8ca8de404275bc63795e17cb197e: stopping tablet replica
I20260809 15:36:06.023806 10626 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:36:06.023466 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.89.253:42075 (local address 127.0.0.1:36476)
0809 15:36:06.023603 (+ 137us) negotiation.cc:107] Waiting for socket to connect
0809 15:36:06.023708 (+ 105us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.89.253:42075: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":50}
W20260809 15:36:06.024042 10001 consensus_peers.cc:487] T 00000000000000000000000000000000 P b7ef4cef863642f19bd9f01ae2792fe3 -> Peer d36c8ca8de404275bc63795e17cb197e (127.3.89.253:42075): Couldn't send request to peer d36c8ca8de404275bc63795e17cb197e. Status: Network error: Client connection negotiation failed: client connection to 127.3.89.253:42075: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260809 15:36:06.031575 10522 master.cc:552] Master@127.3.89.253:42075 shutdown complete.
I20260809 15:36:06.031649 10522 master_runner.cc:305] Clearing existing system tablet
I20260809 15:36:06.031939 10522 fs_manager.cc:359] Metadata directory not provided
I20260809 15:36:06.031992 10522 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.3.89.252 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786289732823074-3431-0/krb5kdc/oryx_127.3.89.252.keytab.
Entry for principal oryx/127.3.89.252 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786289732823074-3431-0/krb5kdc/oryx_127.3.89.252.keytab.
I20260809 15:36:06.033355 10522 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260809 15:36:06.033905 3431 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskm314YR/build/debug/bin/kudu
/tmp/dist-test-taskm314YR/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786289732823074-3431-0/minicluster-data/master-2/wal
--fs_data_dirs=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786289732823074-3431-0/minicluster-data/master-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786289732823074-3431-0/minicluster-data/master-2/logs
--server_dump_info_path=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786289732823074-3431-0/minicluster-data/master-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.3.89.252:44407
--webserver_interface=127.3.89.252
--webserver_port=0
--keytab_file=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786289732823074-3431-0/krb5kdc/oryx_127.3.89.252.keytab
--principal=oryx/127.3.89.252
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.3.89.212:43259
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.3.89.254:46611,127.3.89.253:42075,127.3.89.252:44407
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786289732823074-3431-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786289732823074-3431-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786289732823074-3431-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260809 15:36:06.035784 10636 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 15:36:06.036068 10522 fs_report.cc:383] FS layout report
--------------------
wal directory:
metadata directory:
1 data directories: /tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786289732823074-3431-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 15:36:06.036264 10522 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260809 15:36:06.036327 10522 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786289732823074-3431-0/minicluster-data/master-1/data,/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786289732823074-3431-0/minicluster-data/master-1/wal
uuid: "d36c8ca8de404275bc63795e17cb197e"
format_stamp: "Formatted at 2026-08-09 15:36:05 on dist-test-slave-2w3w"
server_key: "35be02d71884d25fb755ae017044143b"
server_key_iv: "98ba96cce0359522d9c7551d13cfb1e1"
server_key_version: "encryptionkey@0"
I20260809 15:36:06.038436 10522 ts_tablet_manager.cc:1905] T 00000000000000000000000000000000 P d36c8ca8de404275bc63795e17cb197e: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 15:36:06.040674 10522 ts_tablet_manager.cc:1918] T 00000000000000000000000000000000 P d36c8ca8de404275bc63795e17cb197e: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId unknown
I20260809 15:36:06.040792 10522 log.cc:1196] T 00000000000000000000000000000000 P d36c8ca8de404275bc63795e17cb197e: Deleting WAL directory at /tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786289732823074-3431-0/minicluster-data/master-1/wal/wals/00000000000000000000000000000000
I20260809 15:36:06.041100 10522 ts_tablet_manager.cc:1939] T 00000000000000000000000000000000 P d36c8ca8de404275bc63795e17cb197e: Deleting consensus metadata
I20260809 15:36:06.041311 10522 master_runner.cc:315] Copying system tablet from 127.3.89.254:46611
I20260809 15:36:06.042716 10522 tablet_copy_client.cc:316] T 00000000000000000000000000000000 P d36c8ca8de404275bc63795e17cb197e: tablet copy: Beginning tablet copy session from remote peer at address 127.3.89.254:46611
I20260809 15:36:06.054183 10057 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:36:06.043517 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53150 (local address 127.3.89.254:46611)
0809 15:36:06.043664 (+ 147us) server_negotiation.cc:206] Beginning negotiation
0809 15:36:06.043669 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 15:36:06.046399 (+ 2730us) server_negotiation.cc:407] Connection header received
0809 15:36:06.046598 (+ 199us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 15:36:06.046602 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 15:36:06.046670 (+ 68us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 15:36:06.046769 (+ 99us) server_negotiation.cc:226] Negotiated authn=SASL
0809 15:36:06.047376 (+ 607us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:36:06.048024 (+ 648us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:36:06.049035 (+ 1011us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:36:06.049268 (+ 233us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:36:06.049395 (+ 127us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:36:06.050669 (+ 1274us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 15:36:06.050674 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 15:36:06.050676 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 15:36:06.050710 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0809 15:36:06.052386 (+ 1676us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 15:36:06.052880 (+ 494us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 15:36:06.052892 (+ 12us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 15:36:06.052894 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 15:36:06.052974 (+ 80us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 15:36:06.053276 (+ 302us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 15:36:06.053281 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 15:36:06.053283 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 15:36:06.053475 (+ 192us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 15:36:06.053597 (+ 122us) server_negotiation.cc:1035] Waiting for connection context
0809 15:36:06.053883 (+ 286us) server_negotiation.cc:299] Negotiation successful
0809 15:36:06.054003 (+ 120us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":58}
I20260809 15:36:06.054194 10644 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:36:06.043443 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.89.254:46611 (local address 127.0.0.1:53150)
0809 15:36:06.046268 (+ 2825us) negotiation.cc:107] Waiting for socket to connect
0809 15:36:06.046295 (+ 27us) client_negotiation.cc:174] Beginning negotiation
0809 15:36:06.046446 (+ 151us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 15:36:06.046789 (+ 343us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 15:36:06.046792 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 15:36:06.047066 (+ 274us) client_negotiation.cc:189] Negotiated authn=SASL
0809 15:36:06.047231 (+ 165us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:36:06.047240 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:36:06.048146 (+ 906us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:36:06.048152 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:36:06.048826 (+ 674us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:36:06.048835 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:36:06.049398 (+ 563us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:36:06.049402 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:36:06.049542 (+ 140us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:36:06.050139 (+ 597us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 15:36:06.050163 (+ 24us) client_negotiation.cc:656] calling sasl_client_start()
0809 15:36:06.050517 (+ 354us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 15:36:06.052529 (+ 2012us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 15:36:06.052533 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 15:36:06.052535 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 15:36:06.052759 (+ 224us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 15:36:06.053074 (+ 315us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 15:36:06.053079 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 15:36:06.053081 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 15:36:06.053177 (+ 96us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 15:36:06.053584 (+ 407us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 15:36:06.053588 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 15:36:06.053701 (+ 113us) client_negotiation.cc:769] Sending connection context
0809 15:36:06.053823 (+ 122us) client_negotiation.cc:240] Negotiation successful
0809 15:36:06.053990 (+ 167us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":2744,"thread_start_us":95,"threads_started":1}
I20260809 15:36:06.055306 10036 tablet_copy_service.cc:140] P b7ef4cef863642f19bd9f01ae2792fe3: Received BeginTabletCopySession request for tablet 00000000000000000000000000000000 from peer d36c8ca8de404275bc63795e17cb197e ({username='oryx', principal='oryx/127.3.89.253@KRBTEST.COM'} at 127.0.0.1:53150)
I20260809 15:36:06.055480 10036 tablet_copy_service.cc:161] P b7ef4cef863642f19bd9f01ae2792fe3: Beginning new tablet copy session on tablet 00000000000000000000000000000000 from peer d36c8ca8de404275bc63795e17cb197e at {username='oryx', principal='oryx/127.3.89.253@KRBTEST.COM'} at 127.0.0.1:53150: session id = d36c8ca8de404275bc63795e17cb197e-00000000000000000000000000000000
I20260809 15:36:06.057071 10036 tablet_copy_source_session.cc:205] T 00000000000000000000000000000000 P b7ef4cef863642f19bd9f01ae2792fe3: Tablet Copy: opened 0 blocks and 1 log segments
I20260809 15:36:06.058720 10522 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total
I20260809 15:36:06.060868 10522 tablet_copy_client.cc:791] T 00000000000000000000000000000000 P d36c8ca8de404275bc63795e17cb197e: tablet copy: Starting download of 0 data blocks...
I20260809 15:36:06.061100 10522 tablet_copy_client.cc:655] T 00000000000000000000000000000000 P d36c8ca8de404275bc63795e17cb197e: tablet copy: Starting download of 1 WAL segments...
I20260809 15:36:06.062871 10522 tablet_copy_client.cc:531] T 00000000000000000000000000000000 P d36c8ca8de404275bc63795e17cb197e: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260809 15:36:06.064746 10036 tablet_copy_service.cc:342] P b7ef4cef863642f19bd9f01ae2792fe3: Request end of tablet copy session d36c8ca8de404275bc63795e17cb197e-00000000000000000000000000000000 received from {username='oryx', principal='oryx/127.3.89.253@KRBTEST.COM'} at 127.0.0.1:53150
I20260809 15:36:06.064857 10036 tablet_copy_service.cc:433] P b7ef4cef863642f19bd9f01ae2792fe3: ending tablet copy session d36c8ca8de404275bc63795e17cb197e-00000000000000000000000000000000 on tablet 00000000000000000000000000000000 with peer d36c8ca8de404275bc63795e17cb197e
I20260809 15:36:06.071830 10522 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 15:36:06.073491 10645 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 15:36:06.073547 10649 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 15:36:06.074059 10646 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 15:36:06.074343 10522 server_base.cc:1034] running on GCE node
I20260809 15:36:06.074513 10522 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 15:36:06.074946 10522 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 15:36:06.076144 10522 hybrid_clock.cc:648] HybridClock initialized: now 1786289766076128 us; error 34 us; skew 500 ppm
Aug 09 15:36:06 dist-test-slave-2w3w krb5kdc[9964](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786289766, etypes {rep=17 tkt=17 ses=17}, oryx/127.3.89.253@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 15:36:06.077750 10522 init.cc:377] Logged in from keytab as oryx/127.3.89.253@KRBTEST.COM (short username oryx)
I20260809 15:36:06.078267 10522 webserver.cc:460] Webserver started at http://127.3.89.253:36099/ using document root <none> and password file <none>
I20260809 15:36:06.078514 10522 fs_manager.cc:359] Metadata directory not provided
I20260809 15:36:06.078578 10522 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 15:36:06.079985 10522 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260809 15:36:06.080976 10658 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 15:36:06.081241 10522 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260809 15:36:06.081350 10522 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786289732823074-3431-0/minicluster-data/master-1/data,/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786289732823074-3431-0/minicluster-data/master-1/wal
uuid: "d36c8ca8de404275bc63795e17cb197e"
format_stamp: "Formatted at 2026-08-09 15:36:05 on dist-test-slave-2w3w"
server_key: "35be02d71884d25fb755ae017044143b"
server_key_iv: "98ba96cce0359522d9c7551d13cfb1e1"
server_key_version: "encryptionkey@0"
I20260809 15:36:06.081455 10522 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786289732823074-3431-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786289732823074-3431-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786289732823074-3431-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 15:36:06.093463 10522 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 15:36:06.095798 10522 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 15:36:06.101542 10522 rpc_server.cc:307] RPC server started. Bound to: 127.3.89.253:42075
I20260809 15:36:06.102161 10522 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786289732823074-3431-0/minicluster-data/master-1/data/info.pb
I20260809 15:36:06.102818 10713 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.3.89.253:42075 every 8 connection(s)
I20260809 15:36:06.104023 10714 sys_catalog.cc:263] Verifying existing consensus state
I20260809 15:36:06.104905 10714 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d36c8ca8de404275bc63795e17cb197e: Bootstrap starting.
I20260809 15:36:06.114019 10714 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d36c8ca8de404275bc63795e17cb197e: Bootstrap replayed 1/1 log segments. Stats: ops{read=5 overwritten=0 applied=5 ignored=0} inserts{seen=3 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260809 15:36:06.114342 10714 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d36c8ca8de404275bc63795e17cb197e: Bootstrap complete.
I20260809 15:36:06.114876 10714 raft_consensus.cc:348] T 00000000000000000000000000000000 P d36c8ca8de404275bc63795e17cb197e [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "b7ef4cef863642f19bd9f01ae2792fe3" member_type: VOTER last_known_addr { host: "127.3.89.254" port: 46611 } } peers { permanent_uuid: "d36c8ca8de404275bc63795e17cb197e" member_type: NON_VOTER last_known_addr { host: "127.3.89.253" port: 42075 } attrs { promote: true } }
I20260809 15:36:06.115056 10714 raft_consensus.cc:729] T 00000000000000000000000000000000 P d36c8ca8de404275bc63795e17cb197e [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: d36c8ca8de404275bc63795e17cb197e, State: Initialized, Role: LEARNER
I20260809 15:36:06.115212 10714 consensus_queue.cc:260] T 00000000000000000000000000000000 P d36c8ca8de404275bc63795e17cb197e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 5, Last appended: 1.5, Last appended by leader: 5, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "b7ef4cef863642f19bd9f01ae2792fe3" member_type: VOTER last_known_addr { host: "127.3.89.254" port: 46611 } } peers { permanent_uuid: "d36c8ca8de404275bc63795e17cb197e" member_type: NON_VOTER last_known_addr { host: "127.3.89.253" port: 42075 } attrs { promote: true } }
I20260809 15:36:06.115566 10717 sys_catalog.cc:455] T 00000000000000000000000000000000 P d36c8ca8de404275bc63795e17cb197e [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 committed_config { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "b7ef4cef863642f19bd9f01ae2792fe3" member_type: VOTER last_known_addr { host: "127.3.89.254" port: 46611 } } peers { permanent_uuid: "d36c8ca8de404275bc63795e17cb197e" member_type: NON_VOTER last_known_addr { host: "127.3.89.253" port: 42075 } attrs { promote: true } } }
I20260809 15:36:06.115738 10717 sys_catalog.cc:458] T 00000000000000000000000000000000 P d36c8ca8de404275bc63795e17cb197e [sys.catalog]: This master's current role is: LEARNER
I20260809 15:36:06.115738 10714 sys_catalog.cc:564] T 00000000000000000000000000000000 P d36c8ca8de404275bc63795e17cb197e [sys.catalog]: configured and running, proceeding with master startup.
I20260809 15:36:06.119733 10731 catalog_manager.cc:1259] Loaded cluster ID: 1422be81015249aead82021a784a5898
I20260809 15:36:06.119812 10731 catalog_manager.cc:1552] T 00000000000000000000000000000000 P d36c8ca8de404275bc63795e17cb197e: loading cluster ID for follower catalog manager: success
I20260809 15:36:06.121507 10731 catalog_manager.cc:1574] T 00000000000000000000000000000000 P d36c8ca8de404275bc63795e17cb197e: acquiring CA information for follower catalog manager: success
I20260809 15:36:06.122633 10731 catalog_manager.cc:1602] T 00000000000000000000000000000000 P d36c8ca8de404275bc63795e17cb197e: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
W20260809 15:36:06.190686 10637 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 15:36:06.190948 10637 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 15:36:06.191076 10637 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 15:36:06.195225 10637 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 15:36:06.195289 10637 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 15:36:06.195309 10637 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 15:36:06.195326 10637 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 15:36:06.195343 10637 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 15:36:06.200629 10637 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.3.89.212:43259
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786289732823074-3431-0/minicluster-data/master-2/data
--fs_wal_dir=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786289732823074-3431-0/minicluster-data/master-2/wal
--ipki_ca_key_size=768
--master_addresses=127.3.89.254:46611,127.3.89.253:42075,127.3.89.252:44407
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786289732823074-3431-0/krb5kdc/oryx_127.3.89.252.keytab
--principal=oryx/127.3.89.252
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.3.89.252:44407
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786289732823074-3431-0/minicluster-data/master-2/data/info.pb
--webserver_interface=127.3.89.252
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.10637
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786289732823074-3431-0/minicluster-data/master-2/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 09 Aug 2026 14:46:55 UTC on ccycloud.quasar-yswkmf.root.comops.site
build id 4576540
I20260809 15:36:06.201671 10637 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 15:36:06.202688 10637 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 15:36:06.211220 10739 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 15:36:06.211234 10741 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 15:36:06.211267 10737 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 15:36:06.211445 10637 server_base.cc:1034] running on GCE node
I20260809 15:36:06.212121 10637 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 15:36:06.213155 10637 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 15:36:06.214351 10637 hybrid_clock.cc:648] HybridClock initialized: now 1786289766214337 us; error 25 us; skew 500 ppm
Aug 09 15:36:06 dist-test-slave-2w3w krb5kdc[9964](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786289766, etypes {rep=17 tkt=17 ses=17}, oryx/127.3.89.252@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 15:36:06.217437 10637 init.cc:377] Logged in from keytab as oryx/127.3.89.252@KRBTEST.COM (short username oryx)
I20260809 15:36:06.218338 10637 webserver.cc:460] Webserver started at http://127.3.89.252:45635/ using document root <none> and password file <none>
I20260809 15:36:06.218889 10637 fs_manager.cc:359] Metadata directory not provided
I20260809 15:36:06.219002 10637 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 15:36:06.219213 10637 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 15:36:06.221177 10637 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786289732823074-3431-0/minicluster-data/master-2/data/instance:
uuid: "4bb903b269ca47afa0e1d4334950ab94"
format_stamp: "Formatted at 2026-08-09 15:36:06 on dist-test-slave-2w3w"
server_key: "14b9b9324a0fd0aa54e75c8d87dd4fd3"
server_key_iv: "0e16417c64a5c6bbc8c2453ef3eabe91"
server_key_version: "encryptionkey@0"
I20260809 15:36:06.221814 10637 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786289732823074-3431-0/minicluster-data/master-2/wal/instance:
uuid: "4bb903b269ca47afa0e1d4334950ab94"
format_stamp: "Formatted at 2026-08-09 15:36:06 on dist-test-slave-2w3w"
server_key: "14b9b9324a0fd0aa54e75c8d87dd4fd3"
server_key_iv: "0e16417c64a5c6bbc8c2453ef3eabe91"
server_key_version: "encryptionkey@0"
I20260809 15:36:06.225587 10637 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.002s sys 0.003s
I20260809 15:36:06.227900 10749 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 15:36:06.229060 10637 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260809 15:36:06.229254 10637 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786289732823074-3431-0/minicluster-data/master-2/data,/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786289732823074-3431-0/minicluster-data/master-2/wal
uuid: "4bb903b269ca47afa0e1d4334950ab94"
format_stamp: "Formatted at 2026-08-09 15:36:06 on dist-test-slave-2w3w"
server_key: "14b9b9324a0fd0aa54e75c8d87dd4fd3"
server_key_iv: "0e16417c64a5c6bbc8c2453ef3eabe91"
server_key_version: "encryptionkey@0"
I20260809 15:36:06.229378 10637 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786289732823074-3431-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786289732823074-3431-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786289732823074-3431-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 15:36:06.240532 10637 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 15:36:06.243685 10637 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 15:36:06.243876 10637 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 15:36:06.250689 10637 rpc_server.cc:307] RPC server started. Bound to: 127.3.89.252:44407
I20260809 15:36:06.250932 10804 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.3.89.252:44407 every 8 connection(s)
I20260809 15:36:06.251664 10637 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786289732823074-3431-0/minicluster-data/master-2/data/info.pb
I20260809 15:36:06.252104 3431 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskm314YR/build/debug/bin/kudu as pid 10637
I20260809 15:36:06.255195 10805 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 15:36:06.259478 10805 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.3.89.254" port: 46611 } has no permanent_uuid. Determining permanent_uuid...
Aug 09 15:36:06 dist-test-slave-2w3w krb5kdc[9964](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786289764, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.3.89.252@KRBTEST.COM
I20260809 15:36:06.266053 10807 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:36:06.253494 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33300 (local address 127.3.89.252:44407)
0809 15:36:06.253909 (+ 415us) server_negotiation.cc:206] Beginning negotiation
0809 15:36:06.253923 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0809 15:36:06.253962 (+ 39us) server_negotiation.cc:407] Connection header received
0809 15:36:06.254564 (+ 602us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 15:36:06.254578 (+ 14us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 15:36:06.254893 (+ 315us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 15:36:06.255275 (+ 382us) server_negotiation.cc:226] Negotiated authn=SASL
0809 15:36:06.256116 (+ 841us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:36:06.257085 (+ 969us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:36:06.257946 (+ 861us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:36:06.258408 (+ 462us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:36:06.258587 (+ 179us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:36:06.261304 (+ 2717us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 15:36:06.261308 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 15:36:06.261319 (+ 11us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 15:36:06.261372 (+ 53us) server_negotiation.cc:922] calling sasl_server_start()
0809 15:36:06.263262 (+ 1890us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 15:36:06.263700 (+ 438us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 15:36:06.263713 (+ 13us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 15:36:06.263718 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0809 15:36:06.263827 (+ 109us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 15:36:06.264178 (+ 351us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 15:36:06.264182 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 15:36:06.264183 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 15:36:06.264551 (+ 368us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 15:36:06.264784 (+ 233us) server_negotiation.cc:1035] Waiting for connection context
0809 15:36:06.265028 (+ 244us) server_negotiation.cc:299] Negotiation successful
0809 15:36:06.265317 (+ 289us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":268,"thread_start_us":139,"threads_started":1}
Aug 09 15:36:06 dist-test-slave-2w3w krb5kdc[9964](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786289766, etypes {rep=17 tkt=17 ses=17}, oryx/127.3.89.252@KRBTEST.COM for oryx/127.3.89.254@KRBTEST.COM
I20260809 15:36:06.270088 10808 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:36:06.261158 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.89.254:46611 (local address 127.0.0.1:53160)
0809 15:36:06.261586 (+ 428us) negotiation.cc:107] Waiting for socket to connect
0809 15:36:06.261655 (+ 69us) client_negotiation.cc:174] Beginning negotiation
0809 15:36:06.261860 (+ 205us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 15:36:06.262159 (+ 299us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 15:36:06.262164 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 15:36:06.262520 (+ 356us) client_negotiation.cc:189] Negotiated authn=SASL
0809 15:36:06.262629 (+ 109us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:36:06.262635 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:36:06.263496 (+ 861us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:36:06.263501 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:36:06.264259 (+ 758us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:36:06.264269 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:36:06.264748 (+ 479us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:36:06.264753 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:36:06.264909 (+ 156us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:36:06.265429 (+ 520us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 15:36:06.265448 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0809 15:36:06.266894 (+ 1446us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 15:36:06.268664 (+ 1770us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 15:36:06.268670 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 15:36:06.268674 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0809 15:36:06.268888 (+ 214us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 15:36:06.269141 (+ 253us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 15:36:06.269143 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 15:36:06.269145 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 15:36:06.269224 (+ 79us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 15:36:06.269593 (+ 369us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 15:36:06.269598 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 15:36:06.269694 (+ 96us) client_negotiation.cc:769] Sending connection context
0809 15:36:06.269824 (+ 130us) client_negotiation.cc:240] Negotiation successful
0809 15:36:06.269960 (+ 136us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":250,"thread_start_us":146,"threads_started":1}
I20260809 15:36:06.270117 10057 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:36:06.261323 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53160 (local address 127.3.89.254:46611)
0809 15:36:06.261468 (+ 145us) server_negotiation.cc:206] Beginning negotiation
0809 15:36:06.261473 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 15:36:06.261798 (+ 325us) server_negotiation.cc:407] Connection header received
0809 15:36:06.261988 (+ 190us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 15:36:06.261995 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 15:36:06.262039 (+ 44us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 15:36:06.262129 (+ 90us) server_negotiation.cc:226] Negotiated authn=SASL
0809 15:36:06.262766 (+ 637us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:36:06.263381 (+ 615us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:36:06.264380 (+ 999us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:36:06.264605 (+ 225us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:36:06.264721 (+ 116us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:36:06.267066 (+ 2345us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 15:36:06.267069 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 15:36:06.267071 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 15:36:06.267094 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0809 15:36:06.268564 (+ 1470us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 15:36:06.268984 (+ 420us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 15:36:06.268986 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 15:36:06.268987 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 15:36:06.269062 (+ 75us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 15:36:06.269296 (+ 234us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 15:36:06.269298 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 15:36:06.269299 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 15:36:06.269456 (+ 157us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 15:36:06.269536 (+ 80us) server_negotiation.cc:1035] Waiting for connection context
0809 15:36:06.269877 (+ 341us) server_negotiation.cc:299] Negotiation successful
0809 15:36:06.269987 (+ 110us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":61}
I20260809 15:36:06.271296 10805 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.3.89.253" port: 42075 } has no permanent_uuid. Determining permanent_uuid...
Aug 09 15:36:06 dist-test-slave-2w3w krb5kdc[9964](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786289766, etypes {rep=17 tkt=17 ses=17}, oryx/127.3.89.252@KRBTEST.COM for oryx/127.3.89.253@KRBTEST.COM
I20260809 15:36:06.277321 10223 heartbeater.cc:503] Master 127.3.89.254:46611 was elected leader, sending a full tablet report...
I20260809 15:36:06.280287 10808 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:36:06.271787 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.89.253:42075 (local address 127.0.0.1:36488)
0809 15:36:06.271934 (+ 147us) negotiation.cc:107] Waiting for socket to connect
0809 15:36:06.271950 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0809 15:36:06.272030 (+ 80us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 15:36:06.272480 (+ 450us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 15:36:06.272483 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 15:36:06.272663 (+ 180us) client_negotiation.cc:189] Negotiated authn=SASL
0809 15:36:06.272753 (+ 90us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:36:06.272758 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:36:06.273744 (+ 986us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:36:06.273750 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:36:06.274508 (+ 758us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:36:06.274518 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:36:06.275121 (+ 603us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:36:06.275125 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:36:06.275232 (+ 107us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:36:06.275622 (+ 390us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 15:36:06.275633 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0809 15:36:06.276895 (+ 1262us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 15:36:06.279029 (+ 2134us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 15:36:06.279032 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 15:36:06.279034 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 15:36:06.279228 (+ 194us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 15:36:06.279482 (+ 254us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 15:36:06.279484 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 15:36:06.279486 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 15:36:06.279555 (+ 69us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 15:36:06.279918 (+ 363us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 15:36:06.279920 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 15:36:06.280001 (+ 81us) client_negotiation.cc:769] Sending connection context
0809 15:36:06.280080 (+ 79us) client_negotiation.cc:240] Negotiation successful
0809 15:36:06.280177 (+ 97us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":63}
I20260809 15:36:06.280440 10809 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:36:06.271990 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36488 (local address 127.3.89.253:42075)
0809 15:36:06.272251 (+ 261us) server_negotiation.cc:206] Beginning negotiation
0809 15:36:06.272254 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0809 15:36:06.272267 (+ 13us) server_negotiation.cc:407] Connection header received
0809 15:36:06.272307 (+ 40us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 15:36:06.272309 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 15:36:06.272350 (+ 41us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 15:36:06.272466 (+ 116us) server_negotiation.cc:226] Negotiated authn=SASL
0809 15:36:06.272880 (+ 414us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:36:06.273625 (+ 745us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:36:06.274635 (+ 1010us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:36:06.274924 (+ 289us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:36:06.275122 (+ 198us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:36:06.277028 (+ 1906us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 15:36:06.277031 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 15:36:06.277033 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 15:36:06.277056 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0809 15:36:06.278885 (+ 1829us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 15:36:06.279315 (+ 430us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 15:36:06.279318 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 15:36:06.279319 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 15:36:06.279402 (+ 83us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 15:36:06.279633 (+ 231us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 15:36:06.279635 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 15:36:06.279636 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 15:36:06.279839 (+ 203us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 15:36:06.279924 (+ 85us) server_negotiation.cc:1035] Waiting for connection context
0809 15:36:06.280143 (+ 219us) server_negotiation.cc:299] Negotiation successful
0809 15:36:06.280246 (+ 103us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":193,"thread_start_us":91,"threads_started":1}
I20260809 15:36:06.281071 10805 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.3.89.252" port: 44407 } has no permanent_uuid. Determining permanent_uuid...
Aug 09 15:36:06 dist-test-slave-2w3w krb5kdc[9964](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786289766, etypes {rep=17 tkt=17 ses=17}, oryx/127.3.89.252@KRBTEST.COM for oryx/127.3.89.252@KRBTEST.COM
I20260809 15:36:06.289510 10808 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:36:06.281560 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.89.252:44407 (local address 127.0.0.1:33308)
0809 15:36:06.281663 (+ 103us) negotiation.cc:107] Waiting for socket to connect
0809 15:36:06.281678 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0809 15:36:06.281750 (+ 72us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 15:36:06.282041 (+ 291us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 15:36:06.282046 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 15:36:06.282205 (+ 159us) client_negotiation.cc:189] Negotiated authn=SASL
0809 15:36:06.282277 (+ 72us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:36:06.282284 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:36:06.283457 (+ 1173us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:36:06.283460 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:36:06.284009 (+ 549us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:36:06.284017 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:36:06.284427 (+ 410us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:36:06.284429 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:36:06.284490 (+ 61us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:36:06.284843 (+ 353us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 15:36:06.284855 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0809 15:36:06.286084 (+ 1229us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 15:36:06.287765 (+ 1681us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 15:36:06.287769 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 15:36:06.287771 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 15:36:06.288104 (+ 333us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 15:36:06.288433 (+ 329us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 15:36:06.288435 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 15:36:06.288437 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 15:36:06.288523 (+ 86us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 15:36:06.288911 (+ 388us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 15:36:06.288915 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 15:36:06.289005 (+ 90us) client_negotiation.cc:769] Sending connection context
0809 15:36:06.289114 (+ 109us) client_negotiation.cc:240] Negotiation successful
0809 15:36:06.289279 (+ 165us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":41}
I20260809 15:36:06.289510 10807 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:36:06.281625 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33308 (local address 127.3.89.252:44407)
0809 15:36:06.281865 (+ 240us) server_negotiation.cc:206] Beginning negotiation
0809 15:36:06.281868 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0809 15:36:06.281876 (+ 8us) server_negotiation.cc:407] Connection header received
0809 15:36:06.281895 (+ 19us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 15:36:06.281896 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 15:36:06.281922 (+ 26us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 15:36:06.281983 (+ 61us) server_negotiation.cc:226] Negotiated authn=SASL
0809 15:36:06.282380 (+ 397us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:36:06.283347 (+ 967us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:36:06.284108 (+ 761us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:36:06.284331 (+ 223us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:36:06.284406 (+ 75us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:36:06.286212 (+ 1806us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 15:36:06.286216 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 15:36:06.286217 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 15:36:06.286237 (+ 20us) server_negotiation.cc:922] calling sasl_server_start()
0809 15:36:06.287656 (+ 1419us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 15:36:06.288231 (+ 575us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 15:36:06.288238 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 15:36:06.288241 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 15:36:06.288324 (+ 83us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 15:36:06.288615 (+ 291us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 15:36:06.288622 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 15:36:06.288624 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 15:36:06.288802 (+ 178us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 15:36:06.288910 (+ 108us) server_negotiation.cc:1035] Waiting for connection context
0809 15:36:06.289173 (+ 263us) server_negotiation.cc:299] Negotiation successful
0809 15:36:06.289342 (+ 169us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":54,"server-negotiator.queue_time_us":192}
I20260809 15:36:06.293176 10805 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 4bb903b269ca47afa0e1d4334950ab94: Bootstrap starting.
I20260809 15:36:06.295650 10805 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 4bb903b269ca47afa0e1d4334950ab94: Neither blocks nor log segments found. Creating new log.
I20260809 15:36:06.296424 10805 log.cc:824] T 00000000000000000000000000000000 P 4bb903b269ca47afa0e1d4334950ab94: Log is configured to *not* fsync() on all Append() calls
I20260809 15:36:06.298437 10805 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 4bb903b269ca47afa0e1d4334950ab94: No bootstrap required, opened a new log
I20260809 15:36:06.301628 10805 raft_consensus.cc:348] T 00000000000000000000000000000000 P 4bb903b269ca47afa0e1d4334950ab94 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b7ef4cef863642f19bd9f01ae2792fe3" member_type: VOTER last_known_addr { host: "127.3.89.254" port: 46611 } } peers { permanent_uuid: "d36c8ca8de404275bc63795e17cb197e" member_type: VOTER last_known_addr { host: "127.3.89.253" port: 42075 } } peers { permanent_uuid: "4bb903b269ca47afa0e1d4334950ab94" member_type: VOTER last_known_addr { host: "127.3.89.252" port: 44407 } }
I20260809 15:36:06.301838 10805 raft_consensus.cc:374] T 00000000000000000000000000000000 P 4bb903b269ca47afa0e1d4334950ab94 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 15:36:06.301925 10805 raft_consensus.cc:729] T 00000000000000000000000000000000 P 4bb903b269ca47afa0e1d4334950ab94 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4bb903b269ca47afa0e1d4334950ab94, State: Initialized, Role: FOLLOWER
I20260809 15:36:06.302346 10805 consensus_queue.cc:260] T 00000000000000000000000000000000 P 4bb903b269ca47afa0e1d4334950ab94 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b7ef4cef863642f19bd9f01ae2792fe3" member_type: VOTER last_known_addr { host: "127.3.89.254" port: 46611 } } peers { permanent_uuid: "d36c8ca8de404275bc63795e17cb197e" member_type: VOTER last_known_addr { host: "127.3.89.253" port: 42075 } } peers { permanent_uuid: "4bb903b269ca47afa0e1d4334950ab94" member_type: VOTER last_known_addr { host: "127.3.89.252" port: 44407 } }
I20260809 15:36:06.302804 10812 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4bb903b269ca47afa0e1d4334950ab94 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b7ef4cef863642f19bd9f01ae2792fe3" member_type: VOTER last_known_addr { host: "127.3.89.254" port: 46611 } } peers { permanent_uuid: "d36c8ca8de404275bc63795e17cb197e" member_type: VOTER last_known_addr { host: "127.3.89.253" port: 42075 } } peers { permanent_uuid: "4bb903b269ca47afa0e1d4334950ab94" member_type: VOTER last_known_addr { host: "127.3.89.252" port: 44407 } } }
I20260809 15:36:06.302940 10812 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4bb903b269ca47afa0e1d4334950ab94 [sys.catalog]: This master's current role is: FOLLOWER
I20260809 15:36:06.303572 10805 sys_catalog.cc:564] T 00000000000000000000000000000000 P 4bb903b269ca47afa0e1d4334950ab94 [sys.catalog]: configured and running, proceeding with master startup.
W20260809 15:36:06.308189 10825 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 4bb903b269ca47afa0e1d4334950ab94: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260809 15:36:06.308274 10825 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260809 15:36:06.311782 10637 master_runner.cc:428] Detected that this master 4bb903b269ca47afa0e1d4334950ab94 is joining an existing cluster
I20260809 15:36:06.311852 10637 master_runner.cc:432] Initiating AddMaster RPC to add 127.3.89.252:44407
I20260809 15:36:06.319804 3431 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskm314YR/build/debug/bin/kudu with pid 10082
I20260809 15:36:06.327239 10809 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:36:06.314151 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36492 (local address 127.3.89.253:42075)
0809 15:36:06.314496 (+ 345us) server_negotiation.cc:206] Beginning negotiation
0809 15:36:06.314502 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 15:36:06.314896 (+ 394us) server_negotiation.cc:407] Connection header received
0809 15:36:06.315111 (+ 215us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 15:36:06.315114 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 15:36:06.315169 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 15:36:06.315229 (+ 60us) server_negotiation.cc:226] Negotiated authn=SASL
0809 15:36:06.315973 (+ 744us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:36:06.316833 (+ 860us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:36:06.318292 (+ 1459us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:36:06.318662 (+ 370us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:36:06.318736 (+ 74us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:36:06.323666 (+ 4930us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 15:36:06.323672 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 15:36:06.323677 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 15:36:06.323710 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0809 15:36:06.325419 (+ 1709us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 15:36:06.325900 (+ 481us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 15:36:06.325905 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 15:36:06.325907 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 15:36:06.325991 (+ 84us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 15:36:06.326304 (+ 313us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 15:36:06.326309 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 15:36:06.326311 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 15:36:06.326500 (+ 189us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 15:36:06.326618 (+ 118us) server_negotiation.cc:1035] Waiting for connection context
0809 15:36:06.326895 (+ 277us) server_negotiation.cc:299] Negotiation successful
0809 15:36:06.327093 (+ 198us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":261}
I20260809 15:36:06.327265 10832 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:36:06.314058 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.89.253:42075 (local address 127.0.0.1:36492)
0809 15:36:06.314809 (+ 751us) negotiation.cc:107] Waiting for socket to connect
0809 15:36:06.314823 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0809 15:36:06.314982 (+ 159us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 15:36:06.315387 (+ 405us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 15:36:06.315390 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 15:36:06.315663 (+ 273us) client_negotiation.cc:189] Negotiated authn=SASL
0809 15:36:06.315809 (+ 146us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:36:06.315817 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:36:06.317056 (+ 1239us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:36:06.317059 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:36:06.318156 (+ 1097us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:36:06.318167 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:36:06.318935 (+ 768us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:36:06.318941 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:36:06.319109 (+ 168us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:36:06.319761 (+ 652us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 15:36:06.319774 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0809 15:36:06.323514 (+ 3740us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 15:36:06.325534 (+ 2020us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 15:36:06.325539 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 15:36:06.325542 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0809 15:36:06.325791 (+ 249us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 15:36:06.326093 (+ 302us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 15:36:06.326098 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 15:36:06.326100 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 15:36:06.326194 (+ 94us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 15:36:06.326606 (+ 412us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 15:36:06.326610 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 15:36:06.326718 (+ 108us) client_negotiation.cc:769] Sending connection context
0809 15:36:06.326840 (+ 122us) client_negotiation.cc:240] Negotiation successful
0809 15:36:06.327037 (+ 197us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":686,"mutex_wait_us":122,"thread_start_us":112,"threads_started":1}
I20260809 15:36:06.337272 3431 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskm314YR/build/debug/bin/kudu
/tmp/dist-test-taskm314YR/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786289732823074-3431-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786289732823074-3431-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786289732823074-3431-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786289732823074-3431-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.3.89.193:44753
--local_ip_for_outbound_sockets=127.3.89.193
--tserver_master_addrs=127.3.89.254:46611
--webserver_port=34961
--webserver_interface=127.3.89.193
--keytab_file=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786289732823074-3431-0/krb5kdc/oryx_127.3.89.193.keytab
--principal=oryx/127.3.89.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.3.89.212:43259
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_master_addrs=127.3.89.254:46611,127.3.89.253:42075
--tserver_master_addrs=127.3.89.254:46611,127.3.89.253:42075,127.3.89.252:44407 with env {KRB5CCNAME=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786289732823074-3431-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786289732823074-3431-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786289732823074-3431-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260809 15:36:06.340212 10807 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:36:06.314508 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33316 (local address 127.3.89.252:44407)
0809 15:36:06.314613 (+ 105us) server_negotiation.cc:206] Beginning negotiation
0809 15:36:06.314617 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 15:36:06.315144 (+ 527us) server_negotiation.cc:407] Connection header received
0809 15:36:06.315287 (+ 143us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 15:36:06.315290 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 15:36:06.315328 (+ 38us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 15:36:06.315374 (+ 46us) server_negotiation.cc:226] Negotiated authn=SASL
0809 15:36:06.316094 (+ 720us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:36:06.316845 (+ 751us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:36:06.318415 (+ 1570us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:36:06.318736 (+ 321us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:36:06.318811 (+ 75us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:36:06.336678 (+ 17867us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 15:36:06.336682 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 15:36:06.336684 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 15:36:06.336704 (+ 20us) server_negotiation.cc:922] calling sasl_server_start()
0809 15:36:06.338552 (+ 1848us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 15:36:06.339028 (+ 476us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 15:36:06.339031 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 15:36:06.339033 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 15:36:06.339108 (+ 75us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 15:36:06.339355 (+ 247us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 15:36:06.339357 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 15:36:06.339359 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 15:36:06.339551 (+ 192us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 15:36:06.339630 (+ 79us) server_negotiation.cc:1035] Waiting for connection context
0809 15:36:06.339901 (+ 271us) server_negotiation.cc:299] Negotiation successful
0809 15:36:06.340048 (+ 147us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":22,"server-negotiator.queue_time_us":39}
I20260809 15:36:06.340783 10833 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:36:06.314695 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.89.252:44407 (local address 127.0.0.1:33316)
0809 15:36:06.315063 (+ 368us) negotiation.cc:107] Waiting for socket to connect
0809 15:36:06.315077 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0809 15:36:06.315197 (+ 120us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 15:36:06.315470 (+ 273us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 15:36:06.315473 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 15:36:06.315772 (+ 299us) client_negotiation.cc:189] Negotiated authn=SASL
0809 15:36:06.315932 (+ 160us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:36:06.315941 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:36:06.317078 (+ 1137us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:36:06.317083 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:36:06.318297 (+ 1214us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:36:06.318308 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:36:06.318939 (+ 631us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:36:06.318943 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:36:06.319253 (+ 310us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:36:06.335519 (+ 16266us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 15:36:06.335535 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0809 15:36:06.335978 (+ 443us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 15:36:06.338666 (+ 2688us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 15:36:06.338668 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 15:36:06.338670 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 15:36:06.338896 (+ 226us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 15:36:06.339202 (+ 306us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 15:36:06.339205 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 15:36:06.339207 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 15:36:06.339278 (+ 71us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 15:36:06.339671 (+ 393us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 15:36:06.339674 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 15:36:06.339744 (+ 70us) client_negotiation.cc:769] Sending connection context
0809 15:36:06.340484 (+ 740us) client_negotiation.cc:240] Negotiation successful
0809 15:36:06.340624 (+ 140us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":306,"mutex_wait_us":289,"thread_start_us":100,"threads_started":1}
I20260809 15:36:06.342392 10057 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:36:06.314597 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53164 (local address 127.3.89.254:46611)
0809 15:36:06.314737 (+ 140us) server_negotiation.cc:206] Beginning negotiation
0809 15:36:06.314742 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 15:36:06.314757 (+ 15us) server_negotiation.cc:407] Connection header received
0809 15:36:06.314799 (+ 42us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 15:36:06.314802 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 15:36:06.314862 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 15:36:06.314947 (+ 85us) server_negotiation.cc:226] Negotiated authn=SASL
0809 15:36:06.315767 (+ 820us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:36:06.316709 (+ 942us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:36:06.318280 (+ 1571us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:36:06.318655 (+ 375us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:36:06.318736 (+ 81us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:36:06.336687 (+ 17951us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 15:36:06.336695 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 15:36:06.336701 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 15:36:06.336742 (+ 41us) server_negotiation.cc:922] calling sasl_server_start()
0809 15:36:06.339469 (+ 2727us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 15:36:06.340340 (+ 871us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 15:36:06.340344 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 15:36:06.340348 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 15:36:06.340453 (+ 105us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 15:36:06.341116 (+ 663us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 15:36:06.341123 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 15:36:06.341128 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0809 15:36:06.341407 (+ 279us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 15:36:06.341494 (+ 87us) server_negotiation.cc:1035] Waiting for connection context
0809 15:36:06.341961 (+ 467us) server_negotiation.cc:299] Negotiation successful
0809 15:36:06.342128 (+ 167us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":38}
I20260809 15:36:06.343174 10831 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:36:06.314013 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.89.254:46611 (local address 127.0.0.1:53164)
0809 15:36:06.314367 (+ 354us) negotiation.cc:107] Waiting for socket to connect
0809 15:36:06.314386 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0809 15:36:06.314495 (+ 109us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 15:36:06.315110 (+ 615us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 15:36:06.315114 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 15:36:06.315383 (+ 269us) client_negotiation.cc:189] Negotiated authn=SASL
0809 15:36:06.315612 (+ 229us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:36:06.315622 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:36:06.316860 (+ 1238us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:36:06.316866 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:36:06.318117 (+ 1251us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:36:06.318128 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:36:06.319410 (+ 1282us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:36:06.319414 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:36:06.319528 (+ 114us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:36:06.336140 (+ 16612us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 15:36:06.336153 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0809 15:36:06.336514 (+ 361us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 15:36:06.339845 (+ 3331us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 15:36:06.339850 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 15:36:06.339854 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0809 15:36:06.340191 (+ 337us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 15:36:06.340739 (+ 548us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 15:36:06.340742 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 15:36:06.340757 (+ 15us) client_negotiation.cc:760] calling sasl_client_step()
0809 15:36:06.340988 (+ 231us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 15:36:06.341581 (+ 593us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 15:36:06.341588 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 15:36:06.341728 (+ 140us) client_negotiation.cc:769] Sending connection context
0809 15:36:06.342637 (+ 909us) client_negotiation.cc:240] Negotiation successful
0809 15:36:06.342826 (+ 189us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":253,"mutex_wait_us":20,"thread_start_us":119,"threads_started":1}
Aug 09 15:36:06 dist-test-slave-2w3w krb5kdc[9964](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786289764, etypes {rep=17 tkt=17 ses=17}, oryx/127.3.89.254@KRBTEST.COM for oryx/127.3.89.252@KRBTEST.COM
I20260809 15:36:06.357689 10807 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:36:06.347678 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33332 (local address 127.3.89.252:44407)
0809 15:36:06.347780 (+ 102us) server_negotiation.cc:206] Beginning negotiation
0809 15:36:06.347786 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 15:36:06.347832 (+ 46us) server_negotiation.cc:407] Connection header received
0809 15:36:06.348031 (+ 199us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 15:36:06.348034 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 15:36:06.348085 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 15:36:06.348160 (+ 75us) server_negotiation.cc:226] Negotiated authn=SASL
0809 15:36:06.348745 (+ 585us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:36:06.349513 (+ 768us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:36:06.350941 (+ 1428us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:36:06.351392 (+ 451us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:36:06.351500 (+ 108us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:36:06.354103 (+ 2603us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 15:36:06.354107 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 15:36:06.354111 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 15:36:06.354135 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0809 15:36:06.355906 (+ 1771us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 15:36:06.356423 (+ 517us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 15:36:06.356426 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 15:36:06.356428 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 15:36:06.356514 (+ 86us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 15:36:06.356824 (+ 310us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 15:36:06.356828 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 15:36:06.356829 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 15:36:06.357032 (+ 203us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 15:36:06.357130 (+ 98us) server_negotiation.cc:1035] Waiting for connection context
0809 15:36:06.357433 (+ 303us) server_negotiation.cc:299] Negotiation successful
0809 15:36:06.357543 (+ 110us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":40}
I20260809 15:36:06.357692 10626 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:36:06.347434 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.89.252:44407 (local address 127.0.0.1:33332)
0809 15:36:06.347763 (+ 329us) negotiation.cc:107] Waiting for socket to connect
0809 15:36:06.347781 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0809 15:36:06.347887 (+ 106us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 15:36:06.348211 (+ 324us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 15:36:06.348218 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 15:36:06.348454 (+ 236us) client_negotiation.cc:189] Negotiated authn=SASL
0809 15:36:06.348607 (+ 153us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:36:06.348619 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:36:06.349648 (+ 1029us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:36:06.349654 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:36:06.350795 (+ 1141us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:36:06.350810 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:36:06.351530 (+ 720us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:36:06.351538 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:36:06.351687 (+ 149us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:36:06.352194 (+ 507us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 15:36:06.352212 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0809 15:36:06.353868 (+ 1656us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 15:36:06.356035 (+ 2167us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 15:36:06.356039 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 15:36:06.356042 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0809 15:36:06.356309 (+ 267us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 15:36:06.356618 (+ 309us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 15:36:06.356622 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 15:36:06.356624 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 15:36:06.356719 (+ 95us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 15:36:06.357135 (+ 416us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 15:36:06.357138 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 15:36:06.357240 (+ 102us) client_negotiation.cc:769] Sending connection context
0809 15:36:06.357361 (+ 121us) client_negotiation.cc:240] Negotiation successful
0809 15:36:06.357510 (+ 149us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":198}
I20260809 15:36:06.358999 10014 catalog_manager.cc:6979] Initiating ChangeConfig request to add master 127.3.89.252:44407: tablet_id: "00000000000000000000000000000000"
type: ADD_PEER
server {
permanent_uuid: "4bb903b269ca47afa0e1d4334950ab94"
member_type: NON_VOTER
last_known_addr {
host: "127.3.89.252"
port: 44407
}
attrs {
promote: true
}
}
dest_uuid: "b7ef4cef863642f19bd9f01ae2792fe3"
cas_config_opid_index: 5
I20260809 15:36:06.359613 10014 consensus_queue.cc:237] T 00000000000000000000000000000000 P b7ef4cef863642f19bd9f01ae2792fe3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 1.5, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "b7ef4cef863642f19bd9f01ae2792fe3" member_type: VOTER last_known_addr { host: "127.3.89.254" port: 46611 } } peers { permanent_uuid: "d36c8ca8de404275bc63795e17cb197e" member_type: NON_VOTER last_known_addr { host: "127.3.89.253" port: 42075 } attrs { promote: true } } peers { permanent_uuid: "4bb903b269ca47afa0e1d4334950ab94" member_type: NON_VOTER last_known_addr { host: "127.3.89.252" port: 44407 } attrs { promote: true } }
I20260809 15:36:06.360641 10628 raft_consensus.cc:2932] T 00000000000000000000000000000000 P b7ef4cef863642f19bd9f01ae2792fe3 [term 1 LEADER]: Committing config change with OpId 1.6: config changed from index 5 to 6, NON_VOTER 4bb903b269ca47afa0e1d4334950ab94 (127.3.89.252) added. New config: { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "b7ef4cef863642f19bd9f01ae2792fe3" member_type: VOTER last_known_addr { host: "127.3.89.254" port: 46611 } } peers { permanent_uuid: "d36c8ca8de404275bc63795e17cb197e" member_type: NON_VOTER last_known_addr { host: "127.3.89.253" port: 42075 } attrs { promote: true } } peers { permanent_uuid: "4bb903b269ca47afa0e1d4334950ab94" member_type: NON_VOTER last_known_addr { host: "127.3.89.252" port: 44407 } attrs { promote: true } } }
I20260809 15:36:06.362289 10628 catalog_manager.cc:6983] Successfully completed master ChangeConfig request to add master 127.3.89.252:44407
I20260809 15:36:06.362471 10628 sys_catalog.cc:455] T 00000000000000000000000000000000 P b7ef4cef863642f19bd9f01ae2792fe3 [sys.catalog]: SysCatalogTable state changed. Reason: Config change replication complete. Latest consensus state: current_term: 1 leader_uuid: "b7ef4cef863642f19bd9f01ae2792fe3" committed_config { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "b7ef4cef863642f19bd9f01ae2792fe3" member_type: VOTER last_known_addr { host: "127.3.89.254" port: 46611 } } peers { permanent_uuid: "d36c8ca8de404275bc63795e17cb197e" member_type: NON_VOTER last_known_addr { host: "127.3.89.253" port: 42075 } attrs { promote: true } } peers { permanent_uuid: "4bb903b269ca47afa0e1d4334950ab94" member_type: NON_VOTER last_known_addr { host: "127.3.89.252" port: 44407 } attrs { promote: true } } }
I20260809 15:36:06.362579 10628 sys_catalog.cc:458] T 00000000000000000000000000000000 P b7ef4cef863642f19bd9f01ae2792fe3 [sys.catalog]: This master's current role is: LEADER
I20260809 15:36:06.363063 10637 master.cc:537] Master@127.3.89.252:44407 shutting down...
I20260809 15:36:06.363566 10778 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 4bb903b269ca47afa0e1d4334950ab94 [term 0 FOLLOWER]: Advancing to term 1
I20260809 15:36:06.365291 10778 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 4bb903b269ca47afa0e1d4334950ab94 [term 1 FOLLOWER]: Refusing update from remote peer b7ef4cef863642f19bd9f01ae2792fe3: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20260809 15:36:06.366712 10631 consensus_queue.cc:1035] T 00000000000000000000000000000000 P b7ef4cef863642f19bd9f01ae2792fe3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4bb903b269ca47afa0e1d4334950ab94" member_type: NON_VOTER last_known_addr { host: "127.3.89.252" port: 44407 } attrs { promote: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 0, Time since last communication: 0.000s
W20260809 15:36:06.368841 10001 consensus_peers.cc:487] T 00000000000000000000000000000000 P b7ef4cef863642f19bd9f01ae2792fe3 -> Peer 4bb903b269ca47afa0e1d4334950ab94 (127.3.89.252:44407): Couldn't send request to peer 4bb903b269ca47afa0e1d4334950ab94. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on Master. This is attempt 1: this message will repeat every 5th retry.
I20260809 15:36:06.369498 10637 raft_consensus.cc:2229] T 00000000000000000000000000000000 P 4bb903b269ca47afa0e1d4334950ab94 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 15:36:06.369835 10637 raft_consensus.cc:2258] T 00000000000000000000000000000000 P 4bb903b269ca47afa0e1d4334950ab94 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 15:36:06.370010 10637 tablet_replica.cc:330] T 00000000000000000000000000000000 P 4bb903b269ca47afa0e1d4334950ab94: stopping tablet replica
I20260809 15:36:06.373914 10809 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:36:06.366599 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36506 (local address 127.3.89.253:42075)
0809 15:36:06.367940 (+ 1341us) server_negotiation.cc:206] Beginning negotiation
0809 15:36:06.367945 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 15:36:06.368504 (+ 559us) server_negotiation.cc:407] Connection header received
0809 15:36:06.368722 (+ 218us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 15:36:06.368727 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 15:36:06.368812 (+ 85us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 15:36:06.368886 (+ 74us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 15:36:06.369374 (+ 488us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:36:06.370233 (+ 859us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:36:06.372307 (+ 2074us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:36:06.373006 (+ 699us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:36:06.373091 (+ 85us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:36:06.373240 (+ 149us) server_negotiation.cc:1035] Waiting for connection context
0809 15:36:06.373644 (+ 404us) server_negotiation.cc:299] Negotiation successful
0809 15:36:06.373727 (+ 83us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1240}
I20260809 15:36:06.374518 10626 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:36:06.366212 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.89.253:42075 (local address 127.0.0.1:36506)
0809 15:36:06.368420 (+ 2208us) negotiation.cc:107] Waiting for socket to connect
0809 15:36:06.368440 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0809 15:36:06.368583 (+ 143us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 15:36:06.369126 (+ 543us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 15:36:06.369129 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 15:36:06.369147 (+ 18us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 15:36:06.369238 (+ 91us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:36:06.369244 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:36:06.370387 (+ 1143us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:36:06.370390 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:36:06.372154 (+ 1764us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:36:06.372166 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:36:06.373348 (+ 1182us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:36:06.373354 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:36:06.373458 (+ 104us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:36:06.373508 (+ 50us) client_negotiation.cc:769] Sending connection context
0809 15:36:06.374295 (+ 787us) client_negotiation.cc:240] Negotiation successful
0809 15:36:06.374395 (+ 100us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":2131}
I20260809 15:36:06.375268 10637 master.cc:552] Master@127.3.89.252:44407 shutdown complete.
I20260809 15:36:06.375262 10686 raft_consensus.cc:1261] T 00000000000000000000000000000000 P d36c8ca8de404275bc63795e17cb197e [term 1 LEARNER]: Refusing update from remote peer b7ef4cef863642f19bd9f01ae2792fe3: Log matching property violated. Preceding OpId in replica: term: 1 index: 5. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20260809 15:36:06.375381 10637 master_runner.cc:305] Clearing existing system tablet
I20260809 15:36:06.375674 10637 fs_manager.cc:359] Metadata directory not provided
I20260809 15:36:06.375764 10637 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 15:36:06.375707 10631 consensus_queue.cc:1035] T 00000000000000000000000000000000 P b7ef4cef863642f19bd9f01ae2792fe3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d36c8ca8de404275bc63795e17cb197e" member_type: NON_VOTER last_known_addr { host: "127.3.89.253" port: 42075 } attrs { promote: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260809 15:36:06.377760 10686 raft_consensus.cc:2932] T 00000000000000000000000000000000 P d36c8ca8de404275bc63795e17cb197e [term 1 LEARNER]: Committing config change with OpId 1.6: config changed from index 5 to 6, NON_VOTER 4bb903b269ca47afa0e1d4334950ab94 (127.3.89.252) added. New config: { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "b7ef4cef863642f19bd9f01ae2792fe3" member_type: VOTER last_known_addr { host: "127.3.89.254" port: 46611 } } peers { permanent_uuid: "d36c8ca8de404275bc63795e17cb197e" member_type: NON_VOTER last_known_addr { host: "127.3.89.253" port: 42075 } attrs { promote: true } } peers { permanent_uuid: "4bb903b269ca47afa0e1d4334950ab94" member_type: NON_VOTER last_known_addr { host: "127.3.89.252" port: 44407 } attrs { promote: true } } }
I20260809 15:36:06.379566 10637 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.001s sys 0.000s
I20260809 15:36:06.379928 10717 sys_catalog.cc:455] T 00000000000000000000000000000000 P d36c8ca8de404275bc63795e17cb197e [sys.catalog]: SysCatalogTable state changed. Reason: New leader b7ef4cef863642f19bd9f01ae2792fe3. Latest consensus state: current_term: 1 leader_uuid: "b7ef4cef863642f19bd9f01ae2792fe3" committed_config { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "b7ef4cef863642f19bd9f01ae2792fe3" member_type: VOTER last_known_addr { host: "127.3.89.254" port: 46611 } } peers { permanent_uuid: "d36c8ca8de404275bc63795e17cb197e" member_type: NON_VOTER last_known_addr { host: "127.3.89.253" port: 42075 } attrs { promote: true } } peers { permanent_uuid: "4bb903b269ca47afa0e1d4334950ab94" member_type: NON_VOTER last_known_addr { host: "127.3.89.252" port: 44407 } attrs { promote: true } } }
I20260809 15:36:06.380043 10717 sys_catalog.cc:458] T 00000000000000000000000000000000 P d36c8ca8de404275bc63795e17cb197e [sys.catalog]: This master's current role is: LEARNER
I20260809 15:36:06.380152 10717 sys_catalog.cc:455] T 00000000000000000000000000000000 P d36c8ca8de404275bc63795e17cb197e [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "b7ef4cef863642f19bd9f01ae2792fe3" committed_config { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "b7ef4cef863642f19bd9f01ae2792fe3" member_type: VOTER last_known_addr { host: "127.3.89.254" port: 46611 } } peers { permanent_uuid: "d36c8ca8de404275bc63795e17cb197e" member_type: NON_VOTER last_known_addr { host: "127.3.89.253" port: 42075 } attrs { promote: true } } peers { permanent_uuid: "4bb903b269ca47afa0e1d4334950ab94" member_type: NON_VOTER last_known_addr { host: "127.3.89.252" port: 44407 } attrs { promote: true } } }
I20260809 15:36:06.380227 10717 sys_catalog.cc:458] T 00000000000000000000000000000000 P d36c8ca8de404275bc63795e17cb197e [sys.catalog]: This master's current role is: LEARNER
I20260809 15:36:06.380625 10839 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 15:36:06.381006 10631 raft_consensus.cc:1050] T 00000000000000000000000000000000 P b7ef4cef863642f19bd9f01ae2792fe3: attempting to promote NON_VOTER d36c8ca8de404275bc63795e17cb197e to VOTER
I20260809 15:36:06.381054 10637 fs_report.cc:383] FS layout report
--------------------
wal directory:
metadata directory:
1 data directories: /tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786289732823074-3431-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 15:36:06.381191 10637 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260809 15:36:06.381227 10637 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786289732823074-3431-0/minicluster-data/master-2/data,/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786289732823074-3431-0/minicluster-data/master-2/wal
uuid: "4bb903b269ca47afa0e1d4334950ab94"
format_stamp: "Formatted at 2026-08-09 15:36:06 on dist-test-slave-2w3w"
server_key: "14b9b9324a0fd0aa54e75c8d87dd4fd3"
server_key_iv: "0e16417c64a5c6bbc8c2453ef3eabe91"
server_key_version: "encryptionkey@0"
I20260809 15:36:06.381453 10631 consensus_queue.cc:237] T 00000000000000000000000000000000 P b7ef4cef863642f19bd9f01ae2792fe3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 1.6, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "b7ef4cef863642f19bd9f01ae2792fe3" member_type: VOTER last_known_addr { host: "127.3.89.254" port: 46611 } } peers { permanent_uuid: "d36c8ca8de404275bc63795e17cb197e" member_type: VOTER last_known_addr { host: "127.3.89.253" port: 42075 } attrs { promote: false } } peers { permanent_uuid: "4bb903b269ca47afa0e1d4334950ab94" member_type: NON_VOTER last_known_addr { host: "127.3.89.252" port: 44407 } attrs { promote: true } }
I20260809 15:36:06.383188 10626 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:36:06.382826 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.89.252:44407 (local address 127.0.0.1:33342)
0809 15:36:06.383055 (+ 229us) negotiation.cc:107] Waiting for socket to connect
0809 15:36:06.383115 (+ 60us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.89.252:44407: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":158}
I20260809 15:36:06.383348 10637 ts_tablet_manager.cc:1905] T 00000000000000000000000000000000 P 4bb903b269ca47afa0e1d4334950ab94: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 15:36:06.383816 10686 raft_consensus.cc:1261] T 00000000000000000000000000000000 P d36c8ca8de404275bc63795e17cb197e [term 1 LEARNER]: Refusing update from remote peer b7ef4cef863642f19bd9f01ae2792fe3: Log matching property violated. Preceding OpId in replica: term: 1 index: 6. Preceding OpId from leader: term: 1 index: 7. (index mismatch)
I20260809 15:36:06.384047 10626 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:36:06.383786 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.89.252:44407 (local address 127.0.0.1:33344)
0809 15:36:06.383933 (+ 147us) negotiation.cc:107] Waiting for socket to connect
0809 15:36:06.383983 (+ 50us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.89.252:44407: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":86}
W20260809 15:36:06.384166 10001 consensus_peers.cc:487] T 00000000000000000000000000000000 P b7ef4cef863642f19bd9f01ae2792fe3 -> Peer 4bb903b269ca47afa0e1d4334950ab94 (127.3.89.252:44407): Couldn't send request to peer 4bb903b269ca47afa0e1d4334950ab94. Status: Network error: Client connection negotiation failed: client connection to 127.3.89.252:44407: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260809 15:36:06.384282 10628 consensus_queue.cc:1035] T 00000000000000000000000000000000 P b7ef4cef863642f19bd9f01ae2792fe3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d36c8ca8de404275bc63795e17cb197e" member_type: VOTER last_known_addr { host: "127.3.89.253" port: 42075 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20260809 15:36:06.385805 10628 raft_consensus.cc:2932] T 00000000000000000000000000000000 P b7ef4cef863642f19bd9f01ae2792fe3 [term 1 LEADER]: Committing config change with OpId 1.7: config changed from index 6 to 7, d36c8ca8de404275bc63795e17cb197e (127.3.89.253) changed from NON_VOTER to VOTER. New config: { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "b7ef4cef863642f19bd9f01ae2792fe3" member_type: VOTER last_known_addr { host: "127.3.89.254" port: 46611 } } peers { permanent_uuid: "d36c8ca8de404275bc63795e17cb197e" member_type: VOTER last_known_addr { host: "127.3.89.253" port: 42075 } attrs { promote: false } } peers { permanent_uuid: "4bb903b269ca47afa0e1d4334950ab94" member_type: NON_VOTER last_known_addr { host: "127.3.89.252" port: 44407 } attrs { promote: true } } }
I20260809 15:36:06.386384 10637 ts_tablet_manager.cc:1918] T 00000000000000000000000000000000 P 4bb903b269ca47afa0e1d4334950ab94: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId unknown
I20260809 15:36:06.386526 10637 log.cc:1196] T 00000000000000000000000000000000 P 4bb903b269ca47afa0e1d4334950ab94: Deleting WAL directory at /tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786289732823074-3431-0/minicluster-data/master-2/wal/wals/00000000000000000000000000000000
I20260809 15:36:06.386896 10637 ts_tablet_manager.cc:1939] T 00000000000000000000000000000000 P 4bb903b269ca47afa0e1d4334950ab94: Deleting consensus metadata
I20260809 15:36:06.387184 10637 master_runner.cc:315] Copying system tablet from 127.3.89.254:46611
I20260809 15:36:06.387527 10686 raft_consensus.cc:2932] T 00000000000000000000000000000000 P d36c8ca8de404275bc63795e17cb197e [term 1 FOLLOWER]: Committing config change with OpId 1.7: config changed from index 6 to 7, d36c8ca8de404275bc63795e17cb197e (127.3.89.253) changed from NON_VOTER to VOTER. New config: { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "b7ef4cef863642f19bd9f01ae2792fe3" member_type: VOTER last_known_addr { host: "127.3.89.254" port: 46611 } } peers { permanent_uuid: "d36c8ca8de404275bc63795e17cb197e" member_type: VOTER last_known_addr { host: "127.3.89.253" port: 42075 } attrs { promote: false } } peers { permanent_uuid: "4bb903b269ca47afa0e1d4334950ab94" member_type: NON_VOTER last_known_addr { host: "127.3.89.252" port: 44407 } attrs { promote: true } } }
I20260809 15:36:06.388586 10637 tablet_copy_client.cc:316] T 00000000000000000000000000000000 P 4bb903b269ca47afa0e1d4334950ab94: tablet copy: Beginning tablet copy session from remote peer at address 127.3.89.254:46611
I20260809 15:36:06.388947 10717 sys_catalog.cc:455] T 00000000000000000000000000000000 P d36c8ca8de404275bc63795e17cb197e [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "b7ef4cef863642f19bd9f01ae2792fe3" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "b7ef4cef863642f19bd9f01ae2792fe3" member_type: VOTER last_known_addr { host: "127.3.89.254" port: 46611 } } peers { permanent_uuid: "d36c8ca8de404275bc63795e17cb197e" member_type: VOTER last_known_addr { host: "127.3.89.253" port: 42075 } attrs { promote: false } } peers { permanent_uuid: "4bb903b269ca47afa0e1d4334950ab94" member_type: NON_VOTER last_known_addr { host: "127.3.89.252" port: 44407 } attrs { promote: true } } }
I20260809 15:36:06.389058 10717 sys_catalog.cc:458] T 00000000000000000000000000000000 P d36c8ca8de404275bc63795e17cb197e [sys.catalog]: This master's current role is: FOLLOWER
I20260809 15:36:06.389864 10628 sys_catalog.cc:455] T 00000000000000000000000000000000 P b7ef4cef863642f19bd9f01ae2792fe3 [sys.catalog]: SysCatalogTable state changed. Reason: Config change replication complete. Latest consensus state: current_term: 1 leader_uuid: "b7ef4cef863642f19bd9f01ae2792fe3" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "b7ef4cef863642f19bd9f01ae2792fe3" member_type: VOTER last_known_addr { host: "127.3.89.254" port: 46611 } } peers { permanent_uuid: "d36c8ca8de404275bc63795e17cb197e" member_type: VOTER last_known_addr { host: "127.3.89.253" port: 42075 } attrs { promote: false } } peers { permanent_uuid: "4bb903b269ca47afa0e1d4334950ab94" member_type: NON_VOTER last_known_addr { host: "127.3.89.252" port: 44407 } attrs { promote: true } } }
I20260809 15:36:06.389979 10628 sys_catalog.cc:458] T 00000000000000000000000000000000 P b7ef4cef863642f19bd9f01ae2792fe3 [sys.catalog]: This master's current role is: LEADER
I20260809 15:36:06.390146 10628 sys_catalog.cc:455] T 00000000000000000000000000000000 P b7ef4cef863642f19bd9f01ae2792fe3 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "b7ef4cef863642f19bd9f01ae2792fe3" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "b7ef4cef863642f19bd9f01ae2792fe3" member_type: VOTER last_known_addr { host: "127.3.89.254" port: 46611 } } peers { permanent_uuid: "d36c8ca8de404275bc63795e17cb197e" member_type: VOTER last_known_addr { host: "127.3.89.253" port: 42075 } attrs { promote: false } } peers { permanent_uuid: "4bb903b269ca47afa0e1d4334950ab94" member_type: NON_VOTER last_known_addr { host: "127.3.89.252" port: 44407 } attrs { promote: true } } }
I20260809 15:36:06.390230 10628 sys_catalog.cc:458] T 00000000000000000000000000000000 P b7ef4cef863642f19bd9f01ae2792fe3 [sys.catalog]: This master's current role is: LEADER
I20260809 15:36:06.399142 10057 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:36:06.389499 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53174 (local address 127.3.89.254:46611)
0809 15:36:06.389612 (+ 113us) server_negotiation.cc:206] Beginning negotiation
0809 15:36:06.389617 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 15:36:06.390287 (+ 670us) server_negotiation.cc:407] Connection header received
0809 15:36:06.390525 (+ 238us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 15:36:06.390529 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 15:36:06.390582 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 15:36:06.390636 (+ 54us) server_negotiation.cc:226] Negotiated authn=SASL
0809 15:36:06.391331 (+ 695us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:36:06.392163 (+ 832us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:36:06.393750 (+ 1587us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:36:06.394091 (+ 341us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:36:06.394193 (+ 102us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:36:06.395593 (+ 1400us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 15:36:06.395597 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 15:36:06.395600 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 15:36:06.395626 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0809 15:36:06.397315 (+ 1689us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 15:36:06.397820 (+ 505us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 15:36:06.397824 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 15:36:06.397825 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 15:36:06.397915 (+ 90us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 15:36:06.398214 (+ 299us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 15:36:06.398217 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 15:36:06.398219 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 15:36:06.398440 (+ 221us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 15:36:06.398540 (+ 100us) server_negotiation.cc:1035] Waiting for connection context
0809 15:36:06.398844 (+ 304us) server_negotiation.cc:299] Negotiation successful
0809 15:36:06.398994 (+ 150us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":45}
I20260809 15:36:06.399158 10847 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:36:06.389719 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.89.254:46611 (local address 127.0.0.1:53174)
0809 15:36:06.390165 (+ 446us) negotiation.cc:107] Waiting for socket to connect
0809 15:36:06.390180 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0809 15:36:06.390399 (+ 219us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 15:36:06.390804 (+ 405us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 15:36:06.390807 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 15:36:06.391076 (+ 269us) client_negotiation.cc:189] Negotiated authn=SASL
0809 15:36:06.391198 (+ 122us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:36:06.391205 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:36:06.392303 (+ 1098us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:36:06.392307 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:36:06.393544 (+ 1237us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:36:06.393559 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:36:06.394221 (+ 662us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:36:06.394226 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:36:06.394315 (+ 89us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:36:06.394942 (+ 627us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 15:36:06.394987 (+ 45us) client_negotiation.cc:656] calling sasl_client_start()
0809 15:36:06.395441 (+ 454us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 15:36:06.397434 (+ 1993us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 15:36:06.397437 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 15:36:06.397439 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 15:36:06.397702 (+ 263us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 15:36:06.398017 (+ 315us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 15:36:06.398020 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 15:36:06.398022 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 15:36:06.398117 (+ 95us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 15:36:06.398544 (+ 427us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 15:36:06.398547 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 15:36:06.398656 (+ 109us) client_negotiation.cc:769] Sending connection context
0809 15:36:06.398772 (+ 116us) client_negotiation.cc:240] Negotiation successful
0809 15:36:06.398913 (+ 141us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":376,"thread_start_us":68,"threads_started":1}
I20260809 15:36:06.399897 10036 tablet_copy_service.cc:140] P b7ef4cef863642f19bd9f01ae2792fe3: Received BeginTabletCopySession request for tablet 00000000000000000000000000000000 from peer 4bb903b269ca47afa0e1d4334950ab94 ({username='oryx', principal='oryx/127.3.89.252@KRBTEST.COM'} at 127.0.0.1:53174)
I20260809 15:36:06.399991 10036 tablet_copy_service.cc:161] P b7ef4cef863642f19bd9f01ae2792fe3: Beginning new tablet copy session on tablet 00000000000000000000000000000000 from peer 4bb903b269ca47afa0e1d4334950ab94 at {username='oryx', principal='oryx/127.3.89.252@KRBTEST.COM'} at 127.0.0.1:53174: session id = 4bb903b269ca47afa0e1d4334950ab94-00000000000000000000000000000000
I20260809 15:36:06.401072 10036 tablet_copy_source_session.cc:205] T 00000000000000000000000000000000 P b7ef4cef863642f19bd9f01ae2792fe3: Tablet Copy: opened 0 blocks and 1 log segments
I20260809 15:36:06.402446 10637 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total
I20260809 15:36:06.404999 10637 tablet_copy_client.cc:791] T 00000000000000000000000000000000 P 4bb903b269ca47afa0e1d4334950ab94: tablet copy: Starting download of 0 data blocks...
I20260809 15:36:06.405185 10637 tablet_copy_client.cc:655] T 00000000000000000000000000000000 P 4bb903b269ca47afa0e1d4334950ab94: tablet copy: Starting download of 1 WAL segments...
I20260809 15:36:06.406840 10637 tablet_copy_client.cc:531] T 00000000000000000000000000000000 P 4bb903b269ca47afa0e1d4334950ab94: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260809 15:36:06.408834 10036 tablet_copy_service.cc:342] P b7ef4cef863642f19bd9f01ae2792fe3: Request end of tablet copy session 4bb903b269ca47afa0e1d4334950ab94-00000000000000000000000000000000 received from {username='oryx', principal='oryx/127.3.89.252@KRBTEST.COM'} at 127.0.0.1:53174
I20260809 15:36:06.408917 10036 tablet_copy_service.cc:433] P b7ef4cef863642f19bd9f01ae2792fe3: ending tablet copy session 4bb903b269ca47afa0e1d4334950ab94-00000000000000000000000000000000 on tablet 00000000000000000000000000000000 with peer 4bb903b269ca47afa0e1d4334950ab94
I20260809 15:36:06.415889 10637 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 15:36:06.417853 10848 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 15:36:06.417979 10851 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 15:36:06.418244 10849 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 15:36:06.418479 10637 server_base.cc:1034] running on GCE node
I20260809 15:36:06.418664 10637 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 15:36:06.419034 10637 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 15:36:06.420245 10637 hybrid_clock.cc:648] HybridClock initialized: now 1786289766420232 us; error 31 us; skew 500 ppm
Aug 09 15:36:06 dist-test-slave-2w3w krb5kdc[9964](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786289766, etypes {rep=17 tkt=17 ses=17}, oryx/127.3.89.252@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 15:36:06.421993 10637 init.cc:377] Logged in from keytab as oryx/127.3.89.252@KRBTEST.COM (short username oryx)
I20260809 15:36:06.422441 10637 webserver.cc:460] Webserver started at http://127.3.89.252:45427/ using document root <none> and password file <none>
I20260809 15:36:06.422642 10637 fs_manager.cc:359] Metadata directory not provided
I20260809 15:36:06.422688 10637 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 15:36:06.424113 10637 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260809 15:36:06.425237 10859 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 15:36:06.425540 10637 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.000s
I20260809 15:36:06.425647 10637 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786289732823074-3431-0/minicluster-data/master-2/data,/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786289732823074-3431-0/minicluster-data/master-2/wal
uuid: "4bb903b269ca47afa0e1d4334950ab94"
format_stamp: "Formatted at 2026-08-09 15:36:06 on dist-test-slave-2w3w"
server_key: "14b9b9324a0fd0aa54e75c8d87dd4fd3"
server_key_iv: "0e16417c64a5c6bbc8c2453ef3eabe91"
server_key_version: "encryptionkey@0"
I20260809 15:36:06.425840 10637 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786289732823074-3431-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786289732823074-3431-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786289732823074-3431-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 15:36:06.442528 10637 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 15:36:06.445032 10637 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 15:36:06.450465 10637 rpc_server.cc:307] RPC server started. Bound to: 127.3.89.252:44407
I20260809 15:36:06.450497 10917 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.3.89.252:44407 every 8 connection(s)
I20260809 15:36:06.451287 10637 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786289732823074-3431-0/minicluster-data/master-2/data/info.pb
I20260809 15:36:06.453239 10918 sys_catalog.cc:263] Verifying existing consensus state
I20260809 15:36:06.454313 10918 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 4bb903b269ca47afa0e1d4334950ab94: Bootstrap starting.
I20260809 15:36:06.466300 10918 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 4bb903b269ca47afa0e1d4334950ab94: Bootstrap replayed 1/1 log segments. Stats: ops{read=7 overwritten=0 applied=7 ignored=0} inserts{seen=3 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260809 15:36:06.466683 10918 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 4bb903b269ca47afa0e1d4334950ab94: Bootstrap complete.
I20260809 15:36:06.467315 10918 raft_consensus.cc:348] T 00000000000000000000000000000000 P 4bb903b269ca47afa0e1d4334950ab94 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "b7ef4cef863642f19bd9f01ae2792fe3" member_type: VOTER last_known_addr { host: "127.3.89.254" port: 46611 } } peers { permanent_uuid: "d36c8ca8de404275bc63795e17cb197e" member_type: VOTER last_known_addr { host: "127.3.89.253" port: 42075 } attrs { promote: false } } peers { permanent_uuid: "4bb903b269ca47afa0e1d4334950ab94" member_type: NON_VOTER last_known_addr { host: "127.3.89.252" port: 44407 } attrs { promote: true } }
I20260809 15:36:06.467480 10918 raft_consensus.cc:729] T 00000000000000000000000000000000 P 4bb903b269ca47afa0e1d4334950ab94 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 4bb903b269ca47afa0e1d4334950ab94, State: Initialized, Role: LEARNER
I20260809 15:36:06.467612 10918 consensus_queue.cc:260] T 00000000000000000000000000000000 P 4bb903b269ca47afa0e1d4334950ab94 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 7, Last appended: 1.7, Last appended by leader: 7, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "b7ef4cef863642f19bd9f01ae2792fe3" member_type: VOTER last_known_addr { host: "127.3.89.254" port: 46611 } } peers { permanent_uuid: "d36c8ca8de404275bc63795e17cb197e" member_type: VOTER last_known_addr { host: "127.3.89.253" port: 42075 } attrs { promote: false } } peers { permanent_uuid: "4bb903b269ca47afa0e1d4334950ab94" member_type: NON_VOTER last_known_addr { host: "127.3.89.252" port: 44407 } attrs { promote: true } }
I20260809 15:36:06.468014 10922 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4bb903b269ca47afa0e1d4334950ab94 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "b7ef4cef863642f19bd9f01ae2792fe3" member_type: VOTER last_known_addr { host: "127.3.89.254" port: 46611 } } peers { permanent_uuid: "d36c8ca8de404275bc63795e17cb197e" member_type: VOTER last_known_addr { host: "127.3.89.253" port: 42075 } attrs { promote: false } } peers { permanent_uuid: "4bb903b269ca47afa0e1d4334950ab94" member_type: NON_VOTER last_known_addr { host: "127.3.89.252" port: 44407 } attrs { promote: true } } }
I20260809 15:36:06.468111 10918 sys_catalog.cc:564] T 00000000000000000000000000000000 P 4bb903b269ca47afa0e1d4334950ab94 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 15:36:06.468247 10922 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4bb903b269ca47afa0e1d4334950ab94 [sys.catalog]: This master's current role is: LEARNER
I20260809 15:36:06.471908 10936 catalog_manager.cc:1259] Loaded cluster ID: 1422be81015249aead82021a784a5898
I20260809 15:36:06.471966 10936 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 4bb903b269ca47afa0e1d4334950ab94: loading cluster ID for follower catalog manager: success
I20260809 15:36:06.474134 10936 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 4bb903b269ca47afa0e1d4334950ab94: acquiring CA information for follower catalog manager: success
I20260809 15:36:06.475837 10936 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 4bb903b269ca47afa0e1d4334950ab94: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
W20260809 15:36:06.486825 10834 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 15:36:06.487156 10834 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 15:36:06.487304 10834 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 15:36:06.491438 10834 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 15:36:06.491516 10834 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 15:36:06.491601 10834 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.3.89.193
I20260809 15:36:06.496030 10367 heartbeater.cc:503] Master 127.3.89.254:46611 was elected leader, sending a full tablet report...
I20260809 15:36:06.497385 10834 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.3.89.212:43259
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786289732823074-3431-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786289732823074-3431-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786289732823074-3431-0/krb5kdc/oryx_127.3.89.193.keytab
--principal=oryx/127.3.89.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.3.89.193:44753
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786289732823074-3431-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.3.89.193
--webserver_port=34961
--tserver_master_addrs=127.3.89.254:46611,127.3.89.253:42075,127.3.89.252:44407
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.10834
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.3.89.193
--log_dir=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786289732823074-3431-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 09 Aug 2026 14:46:55 UTC on ccycloud.quasar-yswkmf.root.comops.site
build id 4576540
I20260809 15:36:06.498483 10834 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 15:36:06.499564 10834 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 15:36:06.509485 10941 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 15:36:06.509493 10942 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 15:36:06.509596 10834 server_base.cc:1034] running on GCE node
W20260809 15:36:06.509837 10944 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 15:36:06.510277 10834 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 15:36:06.510946 10834 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 15:36:06.512159 10834 hybrid_clock.cc:648] HybridClock initialized: now 1786289766512136 us; error 35 us; skew 500 ppm
Aug 09 15:36:06 dist-test-slave-2w3w krb5kdc[9964](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786289766, etypes {rep=17 tkt=17 ses=17}, oryx/127.3.89.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 15:36:06.514818 10834 init.cc:377] Logged in from keytab as oryx/127.3.89.193@KRBTEST.COM (short username oryx)
I20260809 15:36:06.515743 10834 webserver.cc:460] Webserver started at http://127.3.89.193:34961/ using document root <none> and password file <none>
I20260809 15:36:06.516247 10834 fs_manager.cc:359] Metadata directory not provided
I20260809 15:36:06.516316 10834 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 15:36:06.520170 10834 fs_manager.cc:709] Time spent opening directory manager: real 0.002s user 0.000s sys 0.004s
I20260809 15:36:06.522181 10952 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 15:36:06.523415 10834 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.001s
I20260809 15:36:06.523535 10834 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786289732823074-3431-0/minicluster-data/ts-0/data,/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786289732823074-3431-0/minicluster-data/ts-0/wal
uuid: "5cbab4c3b10e42448c1d2d3cd71ea8be"
format_stamp: "Formatted at 2026-08-09 15:36:05 on dist-test-slave-2w3w"
server_key: "08a21654c0209b1aab0fdcab6b3594a4"
server_key_iv: "c75e0032f961a470d7d1e176549bf51f"
server_key_version: "encryptionkey@0"
I20260809 15:36:06.524070 10834 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786289732823074-3431-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786289732823074-3431-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786289732823074-3431-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 15:36:06.537686 10834 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 15:36:06.540669 10834 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 15:36:06.540822 10834 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 15:36:06.541358 10834 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 15:36:06.542155 10834 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 15:36:06.542203 10834 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 15:36:06.542241 10834 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 15:36:06.542256 10834 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 15:36:06.551874 10834 rpc_server.cc:307] RPC server started. Bound to: 127.3.89.193:44753
I20260809 15:36:06.551904 11077 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.3.89.193:44753 every 8 connection(s)
I20260809 15:36:06.552840 10834 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786289732823074-3431-0/minicluster-data/ts-0/data/info.pb
I20260809 15:36:06.557420 3431 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskm314YR/build/debug/bin/kudu as pid 10834
I20260809 15:36:06.557634 3431 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskm314YR/build/debug/bin/kudu with pid 10228
Aug 09 15:36:06 dist-test-slave-2w3w krb5kdc[9964](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786289766, etypes {rep=17 tkt=17 ses=17}, oryx/127.3.89.193@KRBTEST.COM for oryx/127.3.89.253@KRBTEST.COM
Aug 09 15:36:06 dist-test-slave-2w3w krb5kdc[9964](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786289766, etypes {rep=17 tkt=17 ses=17}, oryx/127.3.89.193@KRBTEST.COM for oryx/127.3.89.252@KRBTEST.COM
Aug 09 15:36:06 dist-test-slave-2w3w krb5kdc[9964](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786289766, etypes {rep=17 tkt=17 ses=17}, oryx/127.3.89.193@KRBTEST.COM for oryx/127.3.89.254@KRBTEST.COM
I20260809 15:36:06.571900 11085 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:36:06.556461 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.89.193:47203 (local address 127.3.89.252:44407)
0809 15:36:06.556712 (+ 251us) server_negotiation.cc:206] Beginning negotiation
0809 15:36:06.556716 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 15:36:06.556730 (+ 14us) server_negotiation.cc:407] Connection header received
0809 15:36:06.557371 (+ 641us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 15:36:06.557374 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 15:36:06.557440 (+ 66us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 15:36:06.557558 (+ 118us) server_negotiation.cc:226] Negotiated authn=SASL
0809 15:36:06.560032 (+ 2474us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:36:06.561637 (+ 1605us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:36:06.563496 (+ 1859us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:36:06.564088 (+ 592us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:36:06.564232 (+ 144us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:36:06.567282 (+ 3050us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 15:36:06.567285 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 15:36:06.567288 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 15:36:06.567329 (+ 41us) server_negotiation.cc:922] calling sasl_server_start()
0809 15:36:06.569295 (+ 1966us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 15:36:06.570097 (+ 802us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 15:36:06.570103 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 15:36:06.570108 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0809 15:36:06.570204 (+ 96us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 15:36:06.570607 (+ 403us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 15:36:06.570610 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 15:36:06.570612 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 15:36:06.570868 (+ 256us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 15:36:06.571022 (+ 154us) server_negotiation.cc:1035] Waiting for connection context
0809 15:36:06.571567 (+ 545us) server_negotiation.cc:299] Negotiation successful
0809 15:36:06.571707 (+ 140us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":176,"thread_start_us":79,"threads_started":1}
I20260809 15:36:06.571900 10809 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:36:06.554652 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.89.193:59305 (local address 127.3.89.253:42075)
0809 15:36:06.554801 (+ 149us) server_negotiation.cc:206] Beginning negotiation
0809 15:36:06.554805 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 15:36:06.556077 (+ 1272us) server_negotiation.cc:407] Connection header received
0809 15:36:06.557307 (+ 1230us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 15:36:06.557313 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 15:36:06.557379 (+ 66us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 15:36:06.557435 (+ 56us) server_negotiation.cc:226] Negotiated authn=SASL
0809 15:36:06.559683 (+ 2248us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:36:06.560418 (+ 735us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:36:06.561997 (+ 1579us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:36:06.562342 (+ 345us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:36:06.562423 (+ 81us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:36:06.567770 (+ 5347us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 15:36:06.567776 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 15:36:06.567788 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 15:36:06.567822 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0809 15:36:06.569535 (+ 1713us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 15:36:06.570156 (+ 621us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 15:36:06.570160 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 15:36:06.570162 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 15:36:06.570251 (+ 89us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 15:36:06.570662 (+ 411us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 15:36:06.570668 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 15:36:06.570671 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 15:36:06.570868 (+ 197us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 15:36:06.571022 (+ 154us) server_negotiation.cc:1035] Waiting for connection context
0809 15:36:06.571584 (+ 562us) server_negotiation.cc:299] Negotiation successful
0809 15:36:06.571714 (+ 130us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":50}
I20260809 15:36:06.572937 11083 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:36:06.555260 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.89.253:42075 (local address 127.3.89.193:59305)
0809 15:36:06.555901 (+ 641us) negotiation.cc:107] Waiting for socket to connect
0809 15:36:06.555964 (+ 63us) client_negotiation.cc:174] Beginning negotiation
0809 15:36:06.557044 (+ 1080us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 15:36:06.557803 (+ 759us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 15:36:06.557812 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 15:36:06.559234 (+ 1422us) client_negotiation.cc:189] Negotiated authn=SASL
0809 15:36:06.559521 (+ 287us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:36:06.559543 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:36:06.560586 (+ 1043us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:36:06.560591 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:36:06.561862 (+ 1271us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:36:06.561877 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:36:06.562623 (+ 746us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:36:06.562627 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:36:06.562824 (+ 197us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:36:06.563684 (+ 860us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 15:36:06.563705 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0809 15:36:06.567560 (+ 3855us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 15:36:06.569654 (+ 2094us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 15:36:06.569659 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 15:36:06.569662 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0809 15:36:06.569998 (+ 336us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 15:36:06.570357 (+ 359us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 15:36:06.570363 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 15:36:06.570368 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0809 15:36:06.570523 (+ 155us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 15:36:06.571042 (+ 519us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 15:36:06.571052 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 15:36:06.571323 (+ 271us) client_negotiation.cc:769] Sending connection context
0809 15:36:06.572317 (+ 994us) client_negotiation.cc:240] Negotiation successful
0809 15:36:06.572495 (+ 178us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":455,"mutex_wait_us":47,"spinlock_wait_cycles":60416,"thread_start_us":143,"threads_started":1}
I20260809 15:36:06.573262 11084 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:36:06.556355 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.89.252:44407 (local address 127.3.89.193:47203)
0809 15:36:06.556645 (+ 290us) negotiation.cc:107] Waiting for socket to connect
0809 15:36:06.556659 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0809 15:36:06.557055 (+ 396us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 15:36:06.557803 (+ 748us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 15:36:06.557814 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 15:36:06.558682 (+ 868us) client_negotiation.cc:189] Negotiated authn=SASL
0809 15:36:06.559835 (+ 1153us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:36:06.559851 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:36:06.561832 (+ 1981us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:36:06.561840 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:36:06.563312 (+ 1472us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:36:06.563329 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:36:06.564405 (+ 1076us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:36:06.564414 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:36:06.564578 (+ 164us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:36:06.565169 (+ 591us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 15:36:06.565195 (+ 26us) client_negotiation.cc:656] calling sasl_client_start()
0809 15:36:06.567066 (+ 1871us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 15:36:06.569457 (+ 2391us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 15:36:06.569466 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 15:36:06.569485 (+ 19us) client_negotiation.cc:760] calling sasl_client_step()
0809 15:36:06.569975 (+ 490us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 15:36:06.570326 (+ 351us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 15:36:06.570332 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 15:36:06.570337 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0809 15:36:06.570502 (+ 165us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 15:36:06.571070 (+ 568us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 15:36:06.571073 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 15:36:06.571320 (+ 247us) client_negotiation.cc:769] Sending connection context
0809 15:36:06.571509 (+ 189us) client_negotiation.cc:240] Negotiation successful
0809 15:36:06.571855 (+ 346us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":228,"mutex_wait_us":127,"thread_start_us":112,"threads_started":1}
I20260809 15:36:06.574533 3431 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskm314YR/build/debug/bin/kudu
/tmp/dist-test-taskm314YR/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786289732823074-3431-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786289732823074-3431-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786289732823074-3431-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786289732823074-3431-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.3.89.194:36959
--local_ip_for_outbound_sockets=127.3.89.194
--tserver_master_addrs=127.3.89.254:46611
--webserver_port=43137
--webserver_interface=127.3.89.194
--keytab_file=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786289732823074-3431-0/krb5kdc/oryx_127.3.89.194.keytab
--principal=oryx/127.3.89.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.3.89.212:43259
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_master_addrs=127.3.89.254:46611,127.3.89.253:42075
--tserver_master_addrs=127.3.89.254:46611,127.3.89.253:42075,127.3.89.252:44407 with env {KRB5CCNAME=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786289732823074-3431-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786289732823074-3431-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786289732823074-3431-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260809 15:36:06.574842 11080 heartbeater.cc:347] Connected to a master server at 127.3.89.253:42075
I20260809 15:36:06.575019 11078 heartbeater.cc:347] Connected to a master server at 127.3.89.252:44407
I20260809 15:36:06.575101 11080 heartbeater.cc:464] Registering TS with master...
I20260809 15:36:06.575114 11078 heartbeater.cc:464] Registering TS with master...
I20260809 15:36:06.575707 11078 heartbeater.cc:511] Master 127.3.89.252:44407 requested a full tablet report, sending...
I20260809 15:36:06.575711 11080 heartbeater.cc:511] Master 127.3.89.253:42075 requested a full tablet report, sending...
I20260809 15:36:06.575788 11082 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:36:06.555260 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.89.254:46611 (local address 127.3.89.193:57733)
0809 15:36:06.555901 (+ 641us) negotiation.cc:107] Waiting for socket to connect
0809 15:36:06.555964 (+ 63us) client_negotiation.cc:174] Beginning negotiation
0809 15:36:06.557044 (+ 1080us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 15:36:06.557803 (+ 759us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 15:36:06.557812 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 15:36:06.558650 (+ 838us) client_negotiation.cc:189] Negotiated authn=SASL
0809 15:36:06.559527 (+ 877us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:36:06.559544 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:36:06.563592 (+ 4048us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:36:06.563596 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:36:06.564449 (+ 853us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:36:06.564464 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:36:06.565272 (+ 808us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:36:06.565278 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:36:06.565386 (+ 108us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:36:06.565814 (+ 428us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 15:36:06.565833 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0809 15:36:06.571488 (+ 5655us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 15:36:06.573947 (+ 2459us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 15:36:06.573950 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 15:36:06.573952 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 15:36:06.574204 (+ 252us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 15:36:06.574547 (+ 343us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 15:36:06.574550 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 15:36:06.574551 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0809 15:36:06.574684 (+ 133us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 15:36:06.575275 (+ 591us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 15:36:06.575278 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 15:36:06.575367 (+ 89us) client_negotiation.cc:769] Sending connection context
0809 15:36:06.575459 (+ 92us) client_negotiation.cc:240] Negotiation successful
0809 15:36:06.575600 (+ 141us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":373,"mutex_wait_us":49,"spinlock_wait_cycles":12544,"thread_start_us":153,"threads_started":1}
I20260809 15:36:06.576270 10057 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:36:06.554652 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.89.193:57733 (local address 127.3.89.254:46611)
0809 15:36:06.554836 (+ 184us) server_negotiation.cc:206] Beginning negotiation
0809 15:36:06.554842 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 15:36:06.556078 (+ 1236us) server_negotiation.cc:407] Connection header received
0809 15:36:06.557365 (+ 1287us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 15:36:06.557369 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 15:36:06.557426 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 15:36:06.557495 (+ 69us) server_negotiation.cc:226] Negotiated authn=SASL
0809 15:36:06.562640 (+ 5145us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:36:06.563445 (+ 805us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:36:06.564604 (+ 1159us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:36:06.564973 (+ 369us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:36:06.565089 (+ 116us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:36:06.571666 (+ 6577us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 15:36:06.571673 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 15:36:06.571679 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 15:36:06.571712 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0809 15:36:06.573824 (+ 2112us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 15:36:06.574341 (+ 517us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 15:36:06.574344 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 15:36:06.574346 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 15:36:06.574441 (+ 95us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 15:36:06.574835 (+ 394us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 15:36:06.574838 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 15:36:06.574840 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 15:36:06.575127 (+ 287us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 15:36:06.575229 (+ 102us) server_negotiation.cc:1035] Waiting for connection context
0809 15:36:06.575999 (+ 770us) server_negotiation.cc:299] Negotiation successful
0809 15:36:06.576126 (+ 127us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":50}
I20260809 15:36:06.577270 11079 heartbeater.cc:347] Connected to a master server at 127.3.89.254:46611
I20260809 15:36:06.577359 11079 heartbeater.cc:464] Registering TS with master...
I20260809 15:36:06.577502 11079 heartbeater.cc:511] Master 127.3.89.254:46611 requested a full tablet report, sending...
I20260809 15:36:06.577574 10879 ts_manager.cc:194] Registered new tserver with Master: 5cbab4c3b10e42448c1d2d3cd71ea8be (127.3.89.193:44753)
I20260809 15:36:06.577600 10675 ts_manager.cc:194] Registered new tserver with Master: 5cbab4c3b10e42448c1d2d3cd71ea8be (127.3.89.193:44753)
I20260809 15:36:06.578320 10014 ts_manager.cc:194] Re-registered known tserver with Master: 5cbab4c3b10e42448c1d2d3cd71ea8be (127.3.89.193:44753)
I20260809 15:36:06.578908 10014 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.3.89.193@KRBTEST.COM'} at 127.3.89.193:57733
W20260809 15:36:06.689191 11086 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 15:36:06.689459 11086 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 15:36:06.689503 11086 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 15:36:06.693499 11086 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 15:36:06.693565 11086 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 15:36:06.693655 11086 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.3.89.194
I20260809 15:36:06.699146 11086 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.3.89.212:43259
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786289732823074-3431-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786289732823074-3431-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786289732823074-3431-0/krb5kdc/oryx_127.3.89.194.keytab
--principal=oryx/127.3.89.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.3.89.194:36959
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786289732823074-3431-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.3.89.194
--webserver_port=43137
--tserver_master_addrs=127.3.89.254:46611,127.3.89.253:42075,127.3.89.252:44407
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.11086
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.3.89.194
--log_dir=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786289732823074-3431-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 09 Aug 2026 14:46:55 UTC on ccycloud.quasar-yswkmf.root.comops.site
build id 4576540
I20260809 15:36:06.700187 11086 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 15:36:06.701198 11086 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 15:36:06.710870 11093 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 15:36:06.710850 11095 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 15:36:06.711040 11086 server_base.cc:1034] running on GCE node
W20260809 15:36:06.710855 11092 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 15:36:06.711620 11086 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 15:36:06.711921 10517 heartbeater.cc:503] Master 127.3.89.254:46611 was elected leader, sending a full tablet report...
I20260809 15:36:06.712379 11086 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 15:36:06.713562 11086 hybrid_clock.cc:648] HybridClock initialized: now 1786289766713534 us; error 45 us; skew 500 ppm
Aug 09 15:36:06 dist-test-slave-2w3w krb5kdc[9964](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786289766, etypes {rep=17 tkt=17 ses=17}, oryx/127.3.89.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 15:36:06.716491 11086 init.cc:377] Logged in from keytab as oryx/127.3.89.194@KRBTEST.COM (short username oryx)
I20260809 15:36:06.717473 11086 webserver.cc:460] Webserver started at http://127.3.89.194:43137/ using document root <none> and password file <none>
I20260809 15:36:06.718035 11086 fs_manager.cc:359] Metadata directory not provided
I20260809 15:36:06.718116 11086 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 15:36:06.721858 11086 fs_manager.cc:709] Time spent opening directory manager: real 0.002s user 0.001s sys 0.001s
I20260809 15:36:06.723923 11102 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 15:36:06.725389 11086 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260809 15:36:06.725538 11086 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786289732823074-3431-0/minicluster-data/ts-1/data,/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786289732823074-3431-0/minicluster-data/ts-1/wal
uuid: "7f9e63d41b3e46d9a25fb76d9463f92c"
format_stamp: "Formatted at 2026-08-09 15:36:05 on dist-test-slave-2w3w"
server_key: "63019c5a0e181d6d87f37c46da0da366"
server_key_iv: "307ca22b513284669e73afc9fe660ff5"
server_key_version: "encryptionkey@0"
I20260809 15:36:06.726075 11086 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786289732823074-3431-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786289732823074-3431-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786289732823074-3431-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 15:36:06.749581 11086 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 15:36:06.753247 11086 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 15:36:06.753491 11086 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 15:36:06.754281 11086 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 15:36:06.755321 11086 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 15:36:06.755373 11086 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 15:36:06.755439 11086 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 15:36:06.755471 11086 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 15:36:06.764252 11086 rpc_server.cc:307] RPC server started. Bound to: 127.3.89.194:36959
I20260809 15:36:06.764282 11229 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.3.89.194:36959 every 8 connection(s)
I20260809 15:36:06.765348 11086 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786289732823074-3431-0/minicluster-data/ts-1/data/info.pb
I20260809 15:36:06.772652 3431 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskm314YR/build/debug/bin/kudu as pid 11086
I20260809 15:36:06.772837 3431 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskm314YR/build/debug/bin/kudu with pid 10373
Aug 09 15:36:06 dist-test-slave-2w3w krb5kdc[9964](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786289766, etypes {rep=17 tkt=17 ses=17}, oryx/127.3.89.194@KRBTEST.COM for oryx/127.3.89.254@KRBTEST.COM
Aug 09 15:36:06 dist-test-slave-2w3w krb5kdc[9964](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786289766, etypes {rep=17 tkt=17 ses=17}, oryx/127.3.89.194@KRBTEST.COM for oryx/127.3.89.252@KRBTEST.COM
Aug 09 15:36:06 dist-test-slave-2w3w krb5kdc[9964](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786289766, etypes {rep=17 tkt=17 ses=17}, oryx/127.3.89.194@KRBTEST.COM for oryx/127.3.89.253@KRBTEST.COM
I20260809 15:36:06.784689 10057 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:36:06.767385 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.89.194:44177 (local address 127.3.89.254:46611)
0809 15:36:06.767500 (+ 115us) server_negotiation.cc:206] Beginning negotiation
0809 15:36:06.767503 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0809 15:36:06.768695 (+ 1192us) server_negotiation.cc:407] Connection header received
0809 15:36:06.770181 (+ 1486us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 15:36:06.770186 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 15:36:06.770271 (+ 85us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 15:36:06.770335 (+ 64us) server_negotiation.cc:226] Negotiated authn=SASL
0809 15:36:06.771801 (+ 1466us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:36:06.772573 (+ 772us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:36:06.774832 (+ 2259us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:36:06.775198 (+ 366us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:36:06.775278 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:36:06.779591 (+ 4313us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 15:36:06.779596 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 15:36:06.779599 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 15:36:06.779636 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0809 15:36:06.781988 (+ 2352us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 15:36:06.782724 (+ 736us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 15:36:06.782728 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 15:36:06.782729 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 15:36:06.782824 (+ 95us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 15:36:06.783301 (+ 477us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 15:36:06.783305 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 15:36:06.783309 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 15:36:06.783560 (+ 251us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 15:36:06.783634 (+ 74us) server_negotiation.cc:1035] Waiting for connection context
0809 15:36:06.784392 (+ 758us) server_negotiation.cc:299] Negotiation successful
0809 15:36:06.784521 (+ 129us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":49}
I20260809 15:36:06.784689 10809 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:36:06.769122 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.89.194:47701 (local address 127.3.89.253:42075)
0809 15:36:06.769279 (+ 157us) server_negotiation.cc:206] Beginning negotiation
0809 15:36:06.769284 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 15:36:06.770067 (+ 783us) server_negotiation.cc:407] Connection header received
0809 15:36:06.770140 (+ 73us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 15:36:06.770145 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 15:36:06.770201 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 15:36:06.770281 (+ 80us) server_negotiation.cc:226] Negotiated authn=SASL
0809 15:36:06.771749 (+ 1468us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:36:06.772574 (+ 825us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:36:06.773896 (+ 1322us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:36:06.774363 (+ 467us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:36:06.774502 (+ 139us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:36:06.780584 (+ 6082us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 15:36:06.780590 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 15:36:06.780596 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 15:36:06.780631 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0809 15:36:06.782555 (+ 1924us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 15:36:06.783177 (+ 622us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 15:36:06.783181 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 15:36:06.783182 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 15:36:06.783287 (+ 105us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 15:36:06.783603 (+ 316us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 15:36:06.783607 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 15:36:06.783609 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 15:36:06.783829 (+ 220us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 15:36:06.783909 (+ 80us) server_negotiation.cc:1035] Waiting for connection context
0809 15:36:06.784389 (+ 480us) server_negotiation.cc:299] Negotiation successful
0809 15:36:06.784521 (+ 132us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":66}
I20260809 15:36:06.784827 11085 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:36:06.767252 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.89.194:48291 (local address 127.3.89.252:44407)
0809 15:36:06.767394 (+ 142us) server_negotiation.cc:206] Beginning negotiation
0809 15:36:06.767399 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 15:36:06.768695 (+ 1296us) server_negotiation.cc:407] Connection header received
0809 15:36:06.770191 (+ 1496us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 15:36:06.770195 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 15:36:06.770266 (+ 71us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 15:36:06.770317 (+ 51us) server_negotiation.cc:226] Negotiated authn=SASL
0809 15:36:06.771739 (+ 1422us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:36:06.772309 (+ 570us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:36:06.774833 (+ 2524us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:36:06.775209 (+ 376us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:36:06.775289 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:36:06.779478 (+ 4189us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 15:36:06.779486 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 15:36:06.779492 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 15:36:06.779533 (+ 41us) server_negotiation.cc:922] calling sasl_server_start()
0809 15:36:06.782004 (+ 2471us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 15:36:06.782789 (+ 785us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 15:36:06.782796 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 15:36:06.782801 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0809 15:36:06.782908 (+ 107us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 15:36:06.783346 (+ 438us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 15:36:06.783352 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 15:36:06.783357 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0809 15:36:06.783608 (+ 251us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 15:36:06.783705 (+ 97us) server_negotiation.cc:1035] Waiting for connection context
0809 15:36:06.784393 (+ 688us) server_negotiation.cc:299] Negotiation successful
0809 15:36:06.784552 (+ 159us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":60}
I20260809 15:36:06.786444 11236 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:36:06.768979 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.89.253:42075 (local address 127.3.89.194:47701)
0809 15:36:06.769349 (+ 370us) negotiation.cc:107] Waiting for socket to connect
0809 15:36:06.769361 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0809 15:36:06.769934 (+ 573us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 15:36:06.770503 (+ 569us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 15:36:06.770512 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 15:36:06.770977 (+ 465us) client_negotiation.cc:189] Negotiated authn=SASL
0809 15:36:06.771586 (+ 609us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:36:06.771603 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:36:06.772721 (+ 1118us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:36:06.772724 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:36:06.773763 (+ 1039us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:36:06.773772 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:36:06.774713 (+ 941us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:36:06.774742 (+ 29us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:36:06.775094 (+ 352us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:36:06.776417 (+ 1323us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 15:36:06.776449 (+ 32us) client_negotiation.cc:656] calling sasl_client_start()
0809 15:36:06.780368 (+ 3919us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 15:36:06.782710 (+ 2342us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 15:36:06.782713 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 15:36:06.782772 (+ 59us) client_negotiation.cc:760] calling sasl_client_step()
0809 15:36:06.783054 (+ 282us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 15:36:06.783400 (+ 346us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 15:36:06.783403 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 15:36:06.783405 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 15:36:06.783490 (+ 85us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 15:36:06.783963 (+ 473us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 15:36:06.783966 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 15:36:06.784090 (+ 124us) client_negotiation.cc:769] Sending connection context
0809 15:36:06.785137 (+ 1047us) client_negotiation.cc:240] Negotiation successful
0809 15:36:06.785487 (+ 350us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":307,"mutex_wait_us":288,"spinlock_wait_cycles":18176,"thread_start_us":139,"threads_started":1}
I20260809 15:36:06.786448 11234 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:36:06.767927 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.89.252:44407 (local address 127.3.89.194:48291)
0809 15:36:06.768513 (+ 586us) negotiation.cc:107] Waiting for socket to connect
0809 15:36:06.768573 (+ 60us) client_negotiation.cc:174] Beginning negotiation
0809 15:36:06.769923 (+ 1350us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 15:36:06.770502 (+ 579us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 15:36:06.770509 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 15:36:06.771001 (+ 492us) client_negotiation.cc:189] Negotiated authn=SASL
0809 15:36:06.771585 (+ 584us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:36:06.771599 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:36:06.772529 (+ 930us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:36:06.772533 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:36:06.774660 (+ 2127us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:36:06.774670 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:36:06.775447 (+ 777us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:36:06.775453 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:36:06.775670 (+ 217us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:36:06.776545 (+ 875us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 15:36:06.776577 (+ 32us) client_negotiation.cc:656] calling sasl_client_start()
0809 15:36:06.779211 (+ 2634us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 15:36:06.782197 (+ 2986us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 15:36:06.782203 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 15:36:06.782208 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0809 15:36:06.782633 (+ 425us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 15:36:06.783075 (+ 442us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 15:36:06.783080 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 15:36:06.783085 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0809 15:36:06.783218 (+ 133us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 15:36:06.783777 (+ 559us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 15:36:06.783782 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 15:36:06.784105 (+ 323us) client_negotiation.cc:769] Sending connection context
0809 15:36:06.785048 (+ 943us) client_negotiation.cc:240] Negotiation successful
0809 15:36:06.785319 (+ 271us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":346,"mutex_wait_us":1129,"spinlock_wait_cycles":126464,"thread_start_us":143,"threads_started":1}
I20260809 15:36:06.786443 11235 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:36:06.767927 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.89.254:46611 (local address 127.3.89.194:44177)
0809 15:36:06.768513 (+ 586us) negotiation.cc:107] Waiting for socket to connect
0809 15:36:06.768573 (+ 60us) client_negotiation.cc:174] Beginning negotiation
0809 15:36:06.769934 (+ 1361us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 15:36:06.770503 (+ 569us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 15:36:06.770511 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 15:36:06.770991 (+ 480us) client_negotiation.cc:189] Negotiated authn=SASL
0809 15:36:06.771586 (+ 595us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:36:06.771603 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:36:06.772721 (+ 1118us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:36:06.772736 (+ 15us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:36:06.774660 (+ 1924us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:36:06.774670 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:36:06.775462 (+ 792us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:36:06.775465 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:36:06.775665 (+ 200us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:36:06.776348 (+ 683us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 15:36:06.776489 (+ 141us) client_negotiation.cc:656] calling sasl_client_start()
0809 15:36:06.779329 (+ 2840us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 15:36:06.782133 (+ 2804us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 15:36:06.782140 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 15:36:06.782159 (+ 19us) client_negotiation.cc:760] calling sasl_client_step()
0809 15:36:06.782609 (+ 450us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 15:36:06.782949 (+ 340us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 15:36:06.782952 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 15:36:06.782954 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 15:36:06.783176 (+ 222us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 15:36:06.783721 (+ 545us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 15:36:06.783727 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 15:36:06.784109 (+ 382us) client_negotiation.cc:769] Sending connection context
0809 15:36:06.784855 (+ 746us) client_negotiation.cc:240] Negotiation successful
0809 15:36:06.785418 (+ 563us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":441,"mutex_wait_us":1386,"spinlock_wait_cycles":154752,"thread_start_us":153,"threads_started":1}
I20260809 15:36:06.787815 11231 heartbeater.cc:347] Connected to a master server at 127.3.89.254:46611
I20260809 15:36:06.788130 11231 heartbeater.cc:464] Registering TS with master...
I20260809 15:36:06.788596 11231 heartbeater.cc:511] Master 127.3.89.254:46611 requested a full tablet report, sending...
I20260809 15:36:06.789065 11232 heartbeater.cc:347] Connected to a master server at 127.3.89.253:42075
I20260809 15:36:06.789166 11232 heartbeater.cc:464] Registering TS with master...
I20260809 15:36:06.789355 11232 heartbeater.cc:511] Master 127.3.89.253:42075 requested a full tablet report, sending...
I20260809 15:36:06.789692 10014 ts_manager.cc:194] Re-registered known tserver with Master: 7f9e63d41b3e46d9a25fb76d9463f92c (127.3.89.194:36959)
I20260809 15:36:06.790555 10014 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.3.89.194@KRBTEST.COM'} at 127.3.89.194:44177
I20260809 15:36:06.791155 11230 heartbeater.cc:347] Connected to a master server at 127.3.89.252:44407
I20260809 15:36:06.791236 11230 heartbeater.cc:464] Registering TS with master...
I20260809 15:36:06.791368 11230 heartbeater.cc:511] Master 127.3.89.252:44407 requested a full tablet report, sending...
I20260809 15:36:06.791384 10675 ts_manager.cc:194] Registered new tserver with Master: 7f9e63d41b3e46d9a25fb76d9463f92c (127.3.89.194:36959)
I20260809 15:36:06.792121 10879 ts_manager.cc:194] Registered new tserver with Master: 7f9e63d41b3e46d9a25fb76d9463f92c (127.3.89.194:36959)
I20260809 15:36:06.792573 3431 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskm314YR/build/debug/bin/kudu
/tmp/dist-test-taskm314YR/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786289732823074-3431-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786289732823074-3431-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786289732823074-3431-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786289732823074-3431-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.3.89.195:36983
--local_ip_for_outbound_sockets=127.3.89.195
--tserver_master_addrs=127.3.89.254:46611
--webserver_port=35055
--webserver_interface=127.3.89.195
--keytab_file=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786289732823074-3431-0/krb5kdc/oryx_127.3.89.195.keytab
--principal=oryx/127.3.89.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.3.89.212:43259
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_master_addrs=127.3.89.254:46611,127.3.89.253:42075
--tserver_master_addrs=127.3.89.254:46611,127.3.89.253:42075,127.3.89.252:44407 with env {KRB5CCNAME=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786289732823074-3431-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786289732823074-3431-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786289732823074-3431-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 15:36:06.905570 11237 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 15:36:06.905963 11237 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 15:36:06.906056 11237 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 15:36:06.910277 11237 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 15:36:06.910337 11237 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 15:36:06.910395 11237 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.3.89.195
I20260809 15:36:06.915632 11237 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.3.89.212:43259
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786289732823074-3431-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786289732823074-3431-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786289732823074-3431-0/krb5kdc/oryx_127.3.89.195.keytab
--principal=oryx/127.3.89.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.3.89.195:36983
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786289732823074-3431-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.3.89.195
--webserver_port=35055
--tserver_master_addrs=127.3.89.254:46611,127.3.89.253:42075,127.3.89.252:44407
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.11237
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.3.89.195
--log_dir=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786289732823074-3431-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 09 Aug 2026 14:46:55 UTC on ccycloud.quasar-yswkmf.root.comops.site
build id 4576540
I20260809 15:36:06.916640 11237 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 15:36:06.917538 11237 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 15:36:06.927188 11242 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 15:36:06.927273 11245 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 15:36:06.927237 11237 server_base.cc:1034] running on GCE node
W20260809 15:36:06.927198 11243 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 15:36:06.927887 11237 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 15:36:06.928491 11237 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 15:36:06.929672 11237 hybrid_clock.cc:648] HybridClock initialized: now 1786289766929658 us; error 40 us; skew 500 ppm
Aug 09 15:36:06 dist-test-slave-2w3w krb5kdc[9964](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786289766, etypes {rep=17 tkt=17 ses=17}, oryx/127.3.89.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 15:36:06.932578 11237 init.cc:377] Logged in from keytab as oryx/127.3.89.195@KRBTEST.COM (short username oryx)
I20260809 15:36:06.933416 11237 webserver.cc:460] Webserver started at http://127.3.89.195:35055/ using document root <none> and password file <none>
I20260809 15:36:06.933981 11237 fs_manager.cc:359] Metadata directory not provided
I20260809 15:36:06.934099 11237 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 15:36:06.937870 11237 fs_manager.cc:709] Time spent opening directory manager: real 0.002s user 0.002s sys 0.000s
I20260809 15:36:06.939988 11252 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 15:36:06.941165 11237 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260809 15:36:06.941309 11237 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786289732823074-3431-0/minicluster-data/ts-2/data,/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786289732823074-3431-0/minicluster-data/ts-2/wal
uuid: "4d0727bc39574ad3b523c62e60bb1615"
format_stamp: "Formatted at 2026-08-09 15:36:05 on dist-test-slave-2w3w"
server_key: "4902f1fe9bbb1ab995c4f5920a6e18e2"
server_key_iv: "d7de3438af7bb5a35662fcd865f4b7ae"
server_key_version: "encryptionkey@0"
I20260809 15:36:06.941841 11237 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786289732823074-3431-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786289732823074-3431-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786289732823074-3431-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 15:36:06.957931 11237 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 15:36:06.961321 11237 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 15:36:06.961511 11237 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 15:36:06.962054 11237 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 15:36:06.962850 11237 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 15:36:06.962898 11237 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 15:36:06.962937 11237 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 15:36:06.962952 11237 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 15:36:06.973831 11237 rpc_server.cc:307] RPC server started. Bound to: 127.3.89.195:36983
I20260809 15:36:06.973850 11377 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.3.89.195:36983 every 8 connection(s)
I20260809 15:36:06.974884 11237 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786289732823074-3431-0/minicluster-data/ts-2/data/info.pb
I20260809 15:36:06.979923 3431 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskm314YR/build/debug/bin/kudu as pid 11237
Aug 09 15:36:06 dist-test-slave-2w3w krb5kdc[9964](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786289766, etypes {rep=17 tkt=17 ses=17}, oryx/127.3.89.195@KRBTEST.COM for oryx/127.3.89.253@KRBTEST.COM
Aug 09 15:36:06 dist-test-slave-2w3w krb5kdc[9964](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786289766, etypes {rep=17 tkt=17 ses=17}, oryx/127.3.89.195@KRBTEST.COM for oryx/127.3.89.254@KRBTEST.COM
Aug 09 15:36:06 dist-test-slave-2w3w krb5kdc[9964](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786289766, etypes {rep=17 tkt=17 ses=17}, oryx/127.3.89.195@KRBTEST.COM for oryx/127.3.89.252@KRBTEST.COM
I20260809 15:36:06.993990 10809 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:36:06.978601 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.89.195:53793 (local address 127.3.89.253:42075)
0809 15:36:06.978726 (+ 125us) server_negotiation.cc:206] Beginning negotiation
0809 15:36:06.978730 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 15:36:06.978873 (+ 143us) server_negotiation.cc:407] Connection header received
0809 15:36:06.979431 (+ 558us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 15:36:06.979434 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 15:36:06.979492 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 15:36:06.979551 (+ 59us) server_negotiation.cc:226] Negotiated authn=SASL
0809 15:36:06.982230 (+ 2679us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:36:06.982996 (+ 766us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:36:06.984957 (+ 1961us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:36:06.985331 (+ 374us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:36:06.985422 (+ 91us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:36:06.989613 (+ 4191us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 15:36:06.989617 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 15:36:06.989620 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 15:36:06.989649 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0809 15:36:06.991409 (+ 1760us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 15:36:06.992044 (+ 635us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 15:36:06.992047 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 15:36:06.992050 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 15:36:06.992137 (+ 87us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 15:36:06.992535 (+ 398us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 15:36:06.992538 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 15:36:06.992540 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 15:36:06.992751 (+ 211us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 15:36:06.992846 (+ 95us) server_negotiation.cc:1035] Waiting for connection context
0809 15:36:06.993597 (+ 751us) server_negotiation.cc:299] Negotiation successful
0809 15:36:06.993743 (+ 146us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":22,"server-negotiator.queue_time_us":53}
I20260809 15:36:06.995760 10057 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:36:06.976829 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.89.195:38149 (local address 127.3.89.254:46611)
0809 15:36:06.977104 (+ 275us) server_negotiation.cc:206] Beginning negotiation
0809 15:36:06.977109 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 15:36:06.978076 (+ 967us) server_negotiation.cc:407] Connection header received
0809 15:36:06.979508 (+ 1432us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 15:36:06.979545 (+ 37us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 15:36:06.979651 (+ 106us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 15:36:06.979749 (+ 98us) server_negotiation.cc:226] Negotiated authn=SASL
0809 15:36:06.981295 (+ 1546us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:36:06.982084 (+ 789us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:36:06.984956 (+ 2872us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:36:06.985316 (+ 360us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:36:06.985413 (+ 97us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:36:06.990262 (+ 4849us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 15:36:06.990266 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 15:36:06.990269 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 15:36:06.990296 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0809 15:36:06.992212 (+ 1916us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 15:36:06.993734 (+ 1522us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 15:36:06.993738 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 15:36:06.993740 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 15:36:06.993846 (+ 106us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 15:36:06.994453 (+ 607us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 15:36:06.994457 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 15:36:06.994459 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 15:36:06.994692 (+ 233us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 15:36:06.994794 (+ 102us) server_negotiation.cc:1035] Waiting for connection context
0809 15:36:06.995455 (+ 661us) server_negotiation.cc:299] Negotiation successful
0809 15:36:06.995594 (+ 139us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":3,"server-negotiator.queue_time_us":178}
I20260809 15:36:06.995906 11384 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:36:06.978467 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.89.253:42075 (local address 127.3.89.195:53793)
0809 15:36:06.978775 (+ 308us) negotiation.cc:107] Waiting for socket to connect
0809 15:36:06.978791 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0809 15:36:06.979175 (+ 384us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 15:36:06.979960 (+ 785us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 15:36:06.979969 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 15:36:06.980482 (+ 513us) client_negotiation.cc:189] Negotiated authn=SASL
0809 15:36:06.981132 (+ 650us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:36:06.981152 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:36:06.983131 (+ 1979us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:36:06.983137 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:36:06.984826 (+ 1689us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:36:06.984835 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:36:06.986336 (+ 1501us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:36:06.986343 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:36:06.986446 (+ 103us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:36:06.986862 (+ 416us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 15:36:06.986876 (+ 14us) client_negotiation.cc:656] calling sasl_client_start()
0809 15:36:06.989443 (+ 2567us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 15:36:06.991535 (+ 2092us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 15:36:06.991541 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 15:36:06.991558 (+ 17us) client_negotiation.cc:760] calling sasl_client_step()
0809 15:36:06.991915 (+ 357us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 15:36:06.992235 (+ 320us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 15:36:06.992238 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 15:36:06.992240 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 15:36:06.992417 (+ 177us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 15:36:06.992942 (+ 525us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 15:36:06.992948 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 15:36:06.993287 (+ 339us) client_negotiation.cc:769] Sending connection context
0809 15:36:06.994226 (+ 939us) client_negotiation.cc:240] Negotiation successful
0809 15:36:06.994621 (+ 395us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":213,"mutex_wait_us":140,"spinlock_wait_cycles":2560,"thread_start_us":119,"threads_started":1}
I20260809 15:36:06.996191 11085 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:36:06.976959 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.89.195:60207 (local address 127.3.89.252:44407)
0809 15:36:06.977183 (+ 224us) server_negotiation.cc:206] Beginning negotiation
0809 15:36:06.977186 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0809 15:36:06.978075 (+ 889us) server_negotiation.cc:407] Connection header received
0809 15:36:06.979431 (+ 1356us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 15:36:06.979434 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 15:36:06.979505 (+ 71us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 15:36:06.979558 (+ 53us) server_negotiation.cc:226] Negotiated authn=SASL
0809 15:36:06.982843 (+ 3285us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:36:06.983631 (+ 788us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:36:06.984955 (+ 1324us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:36:06.985318 (+ 363us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:36:06.985409 (+ 91us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:36:06.992303 (+ 6894us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 15:36:06.992307 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 15:36:06.992311 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 15:36:06.992339 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0809 15:36:06.994181 (+ 1842us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 15:36:06.994766 (+ 585us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 15:36:06.994770 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 15:36:06.994771 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 15:36:06.994868 (+ 97us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 15:36:06.995231 (+ 363us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 15:36:06.995234 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 15:36:06.995236 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 15:36:06.995472 (+ 236us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 15:36:06.995541 (+ 69us) server_negotiation.cc:1035] Waiting for connection context
0809 15:36:06.995907 (+ 366us) server_negotiation.cc:299] Negotiation successful
0809 15:36:06.996020 (+ 113us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":141}
I20260809 15:36:06.996749 11382 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:36:06.977361 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.89.254:46611 (local address 127.3.89.195:38149)
0809 15:36:06.977908 (+ 547us) negotiation.cc:107] Waiting for socket to connect
0809 15:36:06.977970 (+ 62us) client_negotiation.cc:174] Beginning negotiation
0809 15:36:06.979186 (+ 1216us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 15:36:06.979962 (+ 776us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 15:36:06.979970 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 15:36:06.980478 (+ 508us) client_negotiation.cc:189] Negotiated authn=SASL
0809 15:36:06.981131 (+ 653us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:36:06.981151 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:36:06.983137 (+ 1986us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:36:06.983140 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:36:06.984808 (+ 1668us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:36:06.984820 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:36:06.985621 (+ 801us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:36:06.985624 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:36:06.985930 (+ 306us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:36:06.986730 (+ 800us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 15:36:06.986755 (+ 25us) client_negotiation.cc:656] calling sasl_client_start()
0809 15:36:06.989859 (+ 3104us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 15:36:06.992379 (+ 2520us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 15:36:06.992385 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 15:36:06.992389 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0809 15:36:06.993596 (+ 1207us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 15:36:06.993982 (+ 386us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 15:36:06.993985 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 15:36:06.993987 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 15:36:06.994337 (+ 350us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 15:36:06.995150 (+ 813us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 15:36:06.995156 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 15:36:06.995258 (+ 102us) client_negotiation.cc:769] Sending connection context
0809 15:36:06.996466 (+ 1208us) client_negotiation.cc:240] Negotiation successful
0809 15:36:06.996579 (+ 113us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":336,"mutex_wait_us":433,"spinlock_wait_cycles":41728,"thread_start_us":130,"threads_started":1}
I20260809 15:36:06.997016 11383 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:36:06.977361 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.89.252:44407 (local address 127.3.89.195:60207)
0809 15:36:06.977908 (+ 547us) negotiation.cc:107] Waiting for socket to connect
0809 15:36:06.977970 (+ 62us) client_negotiation.cc:174] Beginning negotiation
0809 15:36:06.979175 (+ 1205us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 15:36:06.982127 (+ 2952us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 15:36:06.982130 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 15:36:06.982329 (+ 199us) client_negotiation.cc:189] Negotiated authn=SASL
0809 15:36:06.982455 (+ 126us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:36:06.982465 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:36:06.983767 (+ 1302us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:36:06.983770 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:36:06.984821 (+ 1051us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:36:06.984828 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:36:06.985621 (+ 793us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:36:06.985624 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:36:06.985922 (+ 298us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:36:06.987240 (+ 1318us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 15:36:06.987255 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0809 15:36:06.992125 (+ 4870us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 15:36:06.994292 (+ 2167us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 15:36:06.994295 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 15:36:06.994297 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 15:36:06.994621 (+ 324us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 15:36:06.995012 (+ 391us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 15:36:06.995017 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 15:36:06.995021 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0809 15:36:06.995119 (+ 98us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 15:36:06.995621 (+ 502us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 15:36:06.995624 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 15:36:06.995706 (+ 82us) client_negotiation.cc:769] Sending connection context
0809 15:36:06.996336 (+ 630us) client_negotiation.cc:240] Negotiation successful
0809 15:36:06.996497 (+ 161us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":353,"mutex_wait_us":157,"spinlock_wait_cycles":23296,"thread_start_us":160,"threads_started":1}
I20260809 15:36:06.997370 11380 heartbeater.cc:347] Connected to a master server at 127.3.89.253:42075
I20260809 15:36:06.997624 11380 heartbeater.cc:464] Registering TS with master...
I20260809 15:36:06.998009 11379 heartbeater.cc:347] Connected to a master server at 127.3.89.254:46611
I20260809 15:36:06.998109 11379 heartbeater.cc:464] Registering TS with master...
I20260809 15:36:06.998248 11380 heartbeater.cc:511] Master 127.3.89.253:42075 requested a full tablet report, sending...
I20260809 15:36:06.998308 11379 heartbeater.cc:511] Master 127.3.89.254:46611 requested a full tablet report, sending...
I20260809 15:36:06.999296 11378 heartbeater.cc:347] Connected to a master server at 127.3.89.252:44407
I20260809 15:36:06.999380 11378 heartbeater.cc:464] Registering TS with master...
I20260809 15:36:06.999459 10014 ts_manager.cc:194] Re-registered known tserver with Master: 4d0727bc39574ad3b523c62e60bb1615 (127.3.89.195:36983)
I20260809 15:36:06.999581 11378 heartbeater.cc:511] Master 127.3.89.252:44407 requested a full tablet report, sending...
I20260809 15:36:07.000222 10675 ts_manager.cc:194] Registered new tserver with Master: 4d0727bc39574ad3b523c62e60bb1615 (127.3.89.195:36983)
I20260809 15:36:07.000295 10014 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.3.89.195@KRBTEST.COM'} at 127.3.89.195:38149
I20260809 15:36:07.000690 10879 ts_manager.cc:194] Registered new tserver with Master: 4d0727bc39574ad3b523c62e60bb1615 (127.3.89.195:36983)
I20260809 15:36:07.003046 11391 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:36:06.981970 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36510 (local address 127.3.89.253:42075)
0809 15:36:06.983477 (+ 1507us) server_negotiation.cc:206] Beginning negotiation
0809 15:36:06.983482 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 15:36:06.983499 (+ 17us) server_negotiation.cc:407] Connection header received
0809 15:36:06.983533 (+ 34us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 15:36:06.983536 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 15:36:06.983583 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 15:36:06.983672 (+ 89us) server_negotiation.cc:226] Negotiated authn=SASL
0809 15:36:06.984957 (+ 1285us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:36:06.988119 (+ 3162us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:36:06.989630 (+ 1511us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:36:06.990017 (+ 387us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:36:06.990102 (+ 85us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:36:06.999184 (+ 9082us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 15:36:06.999188 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 15:36:06.999193 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 15:36:06.999223 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0809 15:36:07.001009 (+ 1786us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 15:36:07.001551 (+ 542us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 15:36:07.001555 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 15:36:07.001557 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 15:36:07.001646 (+ 89us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 15:36:07.001967 (+ 321us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 15:36:07.001971 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 15:36:07.001972 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 15:36:07.002186 (+ 214us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 15:36:07.002298 (+ 112us) server_negotiation.cc:1035] Waiting for connection context
0809 15:36:07.002729 (+ 431us) server_negotiation.cc:299] Negotiation successful
0809 15:36:07.002858 (+ 129us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":1633,"server-negotiator.queue_time_us":1019,"thread_start_us":59,"threads_started":1}
I20260809 15:36:07.003345 11390 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:36:06.981934 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53188 (local address 127.3.89.254:46611)
0809 15:36:06.985141 (+ 3207us) server_negotiation.cc:206] Beginning negotiation
0809 15:36:06.985146 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 15:36:06.985158 (+ 12us) server_negotiation.cc:407] Connection header received
0809 15:36:06.985196 (+ 38us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 15:36:06.985198 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 15:36:06.985251 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 15:36:06.985350 (+ 99us) server_negotiation.cc:226] Negotiated authn=SASL
0809 15:36:06.986280 (+ 930us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:36:06.987143 (+ 863us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:36:06.991515 (+ 4372us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:36:06.991871 (+ 356us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:36:06.991985 (+ 114us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:36:06.995820 (+ 3835us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 15:36:06.995825 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 15:36:06.995827 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 15:36:06.995851 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0809 15:36:06.997595 (+ 1744us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 15:36:07.001636 (+ 4041us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 15:36:07.001640 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 15:36:07.001641 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 15:36:07.001728 (+ 87us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 15:36:07.002028 (+ 300us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 15:36:07.002031 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 15:36:07.002032 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 15:36:07.002216 (+ 184us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 15:36:07.002336 (+ 120us) server_negotiation.cc:1035] Waiting for connection context
0809 15:36:07.002798 (+ 462us) server_negotiation.cc:299] Negotiation successful
0809 15:36:07.002919 (+ 121us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":805,"server-negotiator.queue_time_us":3130,"thread_start_us":79,"threads_started":1}
I20260809 15:36:07.003800 11394 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:36:06.982670 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33350 (local address 127.3.89.252:44407)
0809 15:36:06.987370 (+ 4700us) server_negotiation.cc:206] Beginning negotiation
0809 15:36:06.987374 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 15:36:06.987388 (+ 14us) server_negotiation.cc:407] Connection header received
0809 15:36:06.987464 (+ 76us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 15:36:06.987467 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 15:36:06.987515 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 15:36:06.987599 (+ 84us) server_negotiation.cc:226] Negotiated authn=SASL
0809 15:36:06.988494 (+ 895us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:36:06.989214 (+ 720us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:36:06.992095 (+ 2881us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:36:06.992438 (+ 343us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:36:06.992533 (+ 95us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:36:06.994516 (+ 1983us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 15:36:06.994520 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 15:36:06.994522 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 15:36:06.994549 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0809 15:36:06.999217 (+ 4668us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 15:36:07.001659 (+ 2442us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 15:36:07.001663 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 15:36:07.001664 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 15:36:07.001748 (+ 84us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 15:36:07.002037 (+ 289us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 15:36:07.002040 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 15:36:07.002042 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 15:36:07.002231 (+ 189us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 15:36:07.002354 (+ 123us) server_negotiation.cc:1035] Waiting for connection context
0809 15:36:07.003306 (+ 952us) server_negotiation.cc:299] Negotiation successful
0809 15:36:07.003436 (+ 130us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":4621,"thread_start_us":96,"threads_started":1}
I20260809 15:36:07.005033 11396 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:36:06.986138 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33354 (local address 127.3.89.252:44407)
0809 15:36:06.988636 (+ 2498us) server_negotiation.cc:206] Beginning negotiation
0809 15:36:06.988640 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 15:36:06.988651 (+ 11us) server_negotiation.cc:407] Connection header received
0809 15:36:06.988685 (+ 34us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 15:36:06.988688 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 15:36:06.988741 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 15:36:06.988868 (+ 127us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 15:36:06.989166 (+ 298us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:36:07.000709 (+ 11543us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:36:07.003906 (+ 3197us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:36:07.004536 (+ 630us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:36:07.004661 (+ 125us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:36:07.004805 (+ 144us) server_negotiation.cc:1035] Waiting for connection context
0809 15:36:07.004856 (+ 51us) server_negotiation.cc:299] Negotiation successful
0809 15:36:07.004921 (+ 65us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":2433,"thread_start_us":78,"threads_started":1}
I20260809 15:36:07.005033 11395 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:36:06.986079 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.89.252:44407 (local address 127.0.0.1:33354)
0809 15:36:06.988355 (+ 2276us) negotiation.cc:107] Waiting for socket to connect
0809 15:36:06.988369 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0809 15:36:06.988455 (+ 86us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 15:36:06.988961 (+ 506us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 15:36:06.988963 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 15:36:06.988980 (+ 17us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 15:36:06.989064 (+ 84us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:36:06.989069 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:36:07.002364 (+ 13295us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:36:07.002371 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:36:07.003762 (+ 1391us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:36:07.003771 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:36:07.004664 (+ 893us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:36:07.004668 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:36:07.004756 (+ 88us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:36:07.004776 (+ 20us) client_negotiation.cc:769] Sending connection context
0809 15:36:07.004843 (+ 67us) client_negotiation.cc:240] Negotiation successful
0809 15:36:07.004916 (+ 73us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":2204,"thread_start_us":82,"threads_started":1}
I20260809 15:36:07.005793 10890 raft_consensus.cc:1203] T 00000000000000000000000000000000 P 4bb903b269ca47afa0e1d4334950ab94 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.6->[1.7-1.7] Dedup: 1.7->[]
I20260809 15:36:07.006323 11397 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4bb903b269ca47afa0e1d4334950ab94 [sys.catalog]: SysCatalogTable state changed. Reason: New leader b7ef4cef863642f19bd9f01ae2792fe3. Latest consensus state: current_term: 1 leader_uuid: "b7ef4cef863642f19bd9f01ae2792fe3" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "b7ef4cef863642f19bd9f01ae2792fe3" member_type: VOTER last_known_addr { host: "127.3.89.254" port: 46611 } } peers { permanent_uuid: "d36c8ca8de404275bc63795e17cb197e" member_type: VOTER last_known_addr { host: "127.3.89.253" port: 42075 } attrs { promote: false } } peers { permanent_uuid: "4bb903b269ca47afa0e1d4334950ab94" member_type: NON_VOTER last_known_addr { host: "127.3.89.252" port: 44407 } attrs { promote: true } } }
I20260809 15:36:07.006460 11397 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4bb903b269ca47afa0e1d4334950ab94 [sys.catalog]: This master's current role is: LEARNER
I20260809 15:36:07.006537 10014 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:53188:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260809 15:36:07.009656 10014 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260809 15:36:07.012142 10846 sys_catalog.cc:455] T 00000000000000000000000000000000 P b7ef4cef863642f19bd9f01ae2792fe3 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "b7ef4cef863642f19bd9f01ae2792fe3" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "b7ef4cef863642f19bd9f01ae2792fe3" member_type: VOTER last_known_addr { host: "127.3.89.254" port: 46611 } } peers { permanent_uuid: "d36c8ca8de404275bc63795e17cb197e" member_type: VOTER last_known_addr { host: "127.3.89.253" port: 42075 } attrs { promote: false } } peers { permanent_uuid: "4bb903b269ca47afa0e1d4334950ab94" member_type: NON_VOTER last_known_addr { host: "127.3.89.252" port: 44407 } attrs { promote: true } } }
I20260809 15:36:07.012300 10846 sys_catalog.cc:458] T 00000000000000000000000000000000 P b7ef4cef863642f19bd9f01ae2792fe3 [sys.catalog]: This master's current role is: LEADER
I20260809 15:36:07.015585 10846 raft_consensus.cc:1050] T 00000000000000000000000000000000 P b7ef4cef863642f19bd9f01ae2792fe3: attempting to promote NON_VOTER 4bb903b269ca47afa0e1d4334950ab94 to VOTER
I20260809 15:36:07.015899 10846 consensus_queue.cc:237] T 00000000000000000000000000000000 P b7ef4cef863642f19bd9f01ae2792fe3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 1.8, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "b7ef4cef863642f19bd9f01ae2792fe3" member_type: VOTER last_known_addr { host: "127.3.89.254" port: 46611 } } peers { permanent_uuid: "d36c8ca8de404275bc63795e17cb197e" member_type: VOTER last_known_addr { host: "127.3.89.253" port: 42075 } attrs { promote: false } } peers { permanent_uuid: "4bb903b269ca47afa0e1d4334950ab94" member_type: VOTER last_known_addr { host: "127.3.89.252" port: 44407 } attrs { promote: false } }
I20260809 15:36:07.017575 10686 raft_consensus.cc:1261] T 00000000000000000000000000000000 P d36c8ca8de404275bc63795e17cb197e [term 1 FOLLOWER]: Refusing update from remote peer b7ef4cef863642f19bd9f01ae2792fe3: Log matching property violated. Preceding OpId in replica: term: 1 index: 8. Preceding OpId from leader: term: 1 index: 9. (index mismatch)
I20260809 15:36:07.018123 10890 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 4bb903b269ca47afa0e1d4334950ab94 [term 1 LEARNER]: Refusing update from remote peer b7ef4cef863642f19bd9f01ae2792fe3: Log matching property violated. Preceding OpId in replica: term: 1 index: 8. Preceding OpId from leader: term: 1 index: 9. (index mismatch)
I20260809 15:36:07.018623 11401 consensus_queue.cc:1035] T 00000000000000000000000000000000 P b7ef4cef863642f19bd9f01ae2792fe3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4bb903b269ca47afa0e1d4334950ab94" member_type: VOTER last_known_addr { host: "127.3.89.252" port: 44407 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20260809 15:36:07.018790 11400 consensus_queue.cc:1035] T 00000000000000000000000000000000 P b7ef4cef863642f19bd9f01ae2792fe3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d36c8ca8de404275bc63795e17cb197e" member_type: VOTER last_known_addr { host: "127.3.89.253" port: 42075 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20260809 15:36:07.020401 10846 raft_consensus.cc:2932] T 00000000000000000000000000000000 P b7ef4cef863642f19bd9f01ae2792fe3 [term 1 LEADER]: Committing config change with OpId 1.9: config changed from index 7 to 9, 4bb903b269ca47afa0e1d4334950ab94 (127.3.89.252) changed from NON_VOTER to VOTER. New config: { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "b7ef4cef863642f19bd9f01ae2792fe3" member_type: VOTER last_known_addr { host: "127.3.89.254" port: 46611 } } peers { permanent_uuid: "d36c8ca8de404275bc63795e17cb197e" member_type: VOTER last_known_addr { host: "127.3.89.253" port: 42075 } attrs { promote: false } } peers { permanent_uuid: "4bb903b269ca47afa0e1d4334950ab94" member_type: VOTER last_known_addr { host: "127.3.89.252" port: 44407 } attrs { promote: false } } }
I20260809 15:36:07.021001 10686 raft_consensus.cc:2932] T 00000000000000000000000000000000 P d36c8ca8de404275bc63795e17cb197e [term 1 FOLLOWER]: Committing config change with OpId 1.9: config changed from index 7 to 9, 4bb903b269ca47afa0e1d4334950ab94 (127.3.89.252) changed from NON_VOTER to VOTER. New config: { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "b7ef4cef863642f19bd9f01ae2792fe3" member_type: VOTER last_known_addr { host: "127.3.89.254" port: 46611 } } peers { permanent_uuid: "d36c8ca8de404275bc63795e17cb197e" member_type: VOTER last_known_addr { host: "127.3.89.253" port: 42075 } attrs { promote: false } } peers { permanent_uuid: "4bb903b269ca47afa0e1d4334950ab94" member_type: VOTER last_known_addr { host: "127.3.89.252" port: 44407 } attrs { promote: false } } }
I20260809 15:36:07.021996 10846 sys_catalog.cc:455] T 00000000000000000000000000000000 P b7ef4cef863642f19bd9f01ae2792fe3 [sys.catalog]: SysCatalogTable state changed. Reason: Config change replication complete. Latest consensus state: current_term: 1 leader_uuid: "b7ef4cef863642f19bd9f01ae2792fe3" committed_config { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "b7ef4cef863642f19bd9f01ae2792fe3" member_type: VOTER last_known_addr { host: "127.3.89.254" port: 46611 } } peers { permanent_uuid: "d36c8ca8de404275bc63795e17cb197e" member_type: VOTER last_known_addr { host: "127.3.89.253" port: 42075 } attrs { promote: false } } peers { permanent_uuid: "4bb903b269ca47afa0e1d4334950ab94" member_type: VOTER last_known_addr { host: "127.3.89.252" port: 44407 } attrs { promote: false } } }
I20260809 15:36:07.022130 10846 sys_catalog.cc:458] T 00000000000000000000000000000000 P b7ef4cef863642f19bd9f01ae2792fe3 [sys.catalog]: This master's current role is: LEADER
I20260809 15:36:07.022079 11399 sys_catalog.cc:455] T 00000000000000000000000000000000 P b7ef4cef863642f19bd9f01ae2792fe3 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "b7ef4cef863642f19bd9f01ae2792fe3" committed_config { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "b7ef4cef863642f19bd9f01ae2792fe3" member_type: VOTER last_known_addr { host: "127.3.89.254" port: 46611 } } peers { permanent_uuid: "d36c8ca8de404275bc63795e17cb197e" member_type: VOTER last_known_addr { host: "127.3.89.253" port: 42075 } attrs { promote: false } } peers { permanent_uuid: "4bb903b269ca47afa0e1d4334950ab94" member_type: VOTER last_known_addr { host: "127.3.89.252" port: 44407 } attrs { promote: false } } }
I20260809 15:36:07.022197 11399 sys_catalog.cc:458] T 00000000000000000000000000000000 P b7ef4cef863642f19bd9f01ae2792fe3 [sys.catalog]: This master's current role is: LEADER
I20260809 15:36:07.022495 11399 sys_catalog.cc:455] T 00000000000000000000000000000000 P b7ef4cef863642f19bd9f01ae2792fe3 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "b7ef4cef863642f19bd9f01ae2792fe3" committed_config { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "b7ef4cef863642f19bd9f01ae2792fe3" member_type: VOTER last_known_addr { host: "127.3.89.254" port: 46611 } } peers { permanent_uuid: "d36c8ca8de404275bc63795e17cb197e" member_type: VOTER last_known_addr { host: "127.3.89.253" port: 42075 } attrs { promote: false } } peers { permanent_uuid: "4bb903b269ca47afa0e1d4334950ab94" member_type: VOTER last_known_addr { host: "127.3.89.252" port: 44407 } attrs { promote: false } } }
I20260809 15:36:07.022601 11399 sys_catalog.cc:458] T 00000000000000000000000000000000 P b7ef4cef863642f19bd9f01ae2792fe3 [sys.catalog]: This master's current role is: LEADER
I20260809 15:36:07.022563 10890 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 4bb903b269ca47afa0e1d4334950ab94 [term 1 FOLLOWER]: Committing config change with OpId 1.9: config changed from index 7 to 9, 4bb903b269ca47afa0e1d4334950ab94 (127.3.89.252) changed from NON_VOTER to VOTER. New config: { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "b7ef4cef863642f19bd9f01ae2792fe3" member_type: VOTER last_known_addr { host: "127.3.89.254" port: 46611 } } peers { permanent_uuid: "d36c8ca8de404275bc63795e17cb197e" member_type: VOTER last_known_addr { host: "127.3.89.253" port: 42075 } attrs { promote: false } } peers { permanent_uuid: "4bb903b269ca47afa0e1d4334950ab94" member_type: VOTER last_known_addr { host: "127.3.89.252" port: 44407 } attrs { promote: false } } }
I20260809 15:36:07.023350 11409 sys_catalog.cc:455] T 00000000000000000000000000000000 P d36c8ca8de404275bc63795e17cb197e [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "b7ef4cef863642f19bd9f01ae2792fe3" committed_config { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "b7ef4cef863642f19bd9f01ae2792fe3" member_type: VOTER last_known_addr { host: "127.3.89.254" port: 46611 } } peers { permanent_uuid: "d36c8ca8de404275bc63795e17cb197e" member_type: VOTER last_known_addr { host: "127.3.89.253" port: 42075 } attrs { promote: false } } peers { permanent_uuid: "4bb903b269ca47afa0e1d4334950ab94" member_type: VOTER last_known_addr { host: "127.3.89.252" port: 44407 } attrs { promote: false } } }
I20260809 15:36:07.023478 11409 sys_catalog.cc:458] T 00000000000000000000000000000000 P d36c8ca8de404275bc63795e17cb197e [sys.catalog]: This master's current role is: FOLLOWER
I20260809 15:36:07.024647 11397 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4bb903b269ca47afa0e1d4334950ab94 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "b7ef4cef863642f19bd9f01ae2792fe3" committed_config { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "b7ef4cef863642f19bd9f01ae2792fe3" member_type: VOTER last_known_addr { host: "127.3.89.254" port: 46611 } } peers { permanent_uuid: "d36c8ca8de404275bc63795e17cb197e" member_type: VOTER last_known_addr { host: "127.3.89.253" port: 42075 } attrs { promote: false } } peers { permanent_uuid: "4bb903b269ca47afa0e1d4334950ab94" member_type: VOTER last_known_addr { host: "127.3.89.252" port: 44407 } attrs { promote: false } } }
I20260809 15:36:07.024749 11397 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4bb903b269ca47afa0e1d4334950ab94 [sys.catalog]: This master's current role is: FOLLOWER
I20260809 15:36:07.032311 11395 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:36:07.025391 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.89.195:36983 (local address 127.0.0.1:39280)
0809 15:36:07.025503 (+ 112us) negotiation.cc:107] Waiting for socket to connect
0809 15:36:07.025520 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0809 15:36:07.025602 (+ 82us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 15:36:07.027803 (+ 2201us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 15:36:07.027808 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 15:36:07.027830 (+ 22us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 15:36:07.028017 (+ 187us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:36:07.028025 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:36:07.029334 (+ 1309us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:36:07.029338 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:36:07.030715 (+ 1377us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:36:07.030726 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:36:07.031867 (+ 1141us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:36:07.031871 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:36:07.031956 (+ 85us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:36:07.031993 (+ 37us) client_negotiation.cc:769] Sending connection context
0809 15:36:07.032084 (+ 91us) client_negotiation.cc:240] Negotiation successful
0809 15:36:07.032171 (+ 87us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":37,"mutex_wait_us":158}
I20260809 15:36:07.032409 11411 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:36:07.025569 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39280 (local address 127.3.89.195:36983)
0809 15:36:07.027244 (+ 1675us) server_negotiation.cc:206] Beginning negotiation
0809 15:36:07.027257 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0809 15:36:07.027293 (+ 36us) server_negotiation.cc:407] Connection header received
0809 15:36:07.027374 (+ 81us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 15:36:07.027380 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 15:36:07.027646 (+ 266us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 15:36:07.027816 (+ 170us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 15:36:07.028126 (+ 310us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:36:07.029184 (+ 1058us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:36:07.030849 (+ 1665us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:36:07.031740 (+ 891us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:36:07.031848 (+ 108us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:36:07.031998 (+ 150us) server_negotiation.cc:1035] Waiting for connection context
0809 15:36:07.032114 (+ 116us) server_negotiation.cc:299] Negotiation successful
0809 15:36:07.032217 (+ 103us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1578,"thread_start_us":126,"threads_started":1}
I20260809 15:36:07.033008 11414 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:36:07.025709 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.89.193:44753 (local address 127.0.0.1:51608)
0809 15:36:07.027275 (+ 1566us) negotiation.cc:107] Waiting for socket to connect
0809 15:36:07.027286 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0809 15:36:07.027402 (+ 116us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 15:36:07.027855 (+ 453us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 15:36:07.027858 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 15:36:07.027875 (+ 17us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 15:36:07.028046 (+ 171us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:36:07.028054 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:36:07.029592 (+ 1538us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:36:07.029595 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:36:07.031317 (+ 1722us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:36:07.031328 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:36:07.032627 (+ 1299us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:36:07.032630 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:36:07.032697 (+ 67us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:36:07.032712 (+ 15us) client_negotiation.cc:769] Sending connection context
0809 15:36:07.032807 (+ 95us) client_negotiation.cc:240] Negotiation successful
0809 15:36:07.032879 (+ 72us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1504,"mutex_wait_us":72,"thread_start_us":122,"threads_started":1}
I20260809 15:36:07.033082 11413 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:36:07.026144 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51608 (local address 127.3.89.193:44753)
0809 15:36:07.026806 (+ 662us) server_negotiation.cc:206] Beginning negotiation
0809 15:36:07.026829 (+ 23us) server_negotiation.cc:399] Waiting for connection header
0809 15:36:07.027377 (+ 548us) server_negotiation.cc:407] Connection header received
0809 15:36:07.027537 (+ 160us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 15:36:07.027543 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 15:36:07.027711 (+ 168us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 15:36:07.027875 (+ 164us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 15:36:07.028188 (+ 313us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:36:07.029392 (+ 1204us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:36:07.031460 (+ 2068us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:36:07.032240 (+ 780us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:36:07.032319 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:36:07.032451 (+ 132us) server_negotiation.cc:1035] Waiting for connection context
0809 15:36:07.032842 (+ 391us) server_negotiation.cc:299] Negotiation successful
0809 15:36:07.032933 (+ 91us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":548,"thread_start_us":130,"threads_started":1}
I20260809 15:36:07.034003 11415 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:36:07.026947 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37016 (local address 127.3.89.194:36959)
0809 15:36:07.027382 (+ 435us) server_negotiation.cc:206] Beginning negotiation
0809 15:36:07.027392 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0809 15:36:07.028241 (+ 849us) server_negotiation.cc:407] Connection header received
0809 15:36:07.028351 (+ 110us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 15:36:07.028356 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 15:36:07.028537 (+ 181us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 15:36:07.028680 (+ 143us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 15:36:07.028924 (+ 244us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:36:07.030027 (+ 1103us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:36:07.031434 (+ 1407us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:36:07.032118 (+ 684us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:36:07.032214 (+ 96us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:36:07.032347 (+ 133us) server_negotiation.cc:1035] Waiting for connection context
0809 15:36:07.033795 (+ 1448us) server_negotiation.cc:299] Negotiation successful
0809 15:36:07.033890 (+ 95us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":355,"thread_start_us":101,"threads_started":1}
I20260809 15:36:07.034278 11412 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:36:07.026997 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.89.194:36959 (local address 127.0.0.1:37016)
0809 15:36:07.028045 (+ 1048us) negotiation.cc:107] Waiting for socket to connect
0809 15:36:07.028059 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0809 15:36:07.028146 (+ 87us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 15:36:07.028678 (+ 532us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 15:36:07.028681 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 15:36:07.028700 (+ 19us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 15:36:07.028781 (+ 81us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:36:07.028786 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:36:07.030140 (+ 1354us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:36:07.030142 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:36:07.031313 (+ 1171us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:36:07.031322 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:36:07.032412 (+ 1090us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:36:07.032415 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:36:07.032786 (+ 371us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:36:07.032813 (+ 27us) client_negotiation.cc:769] Sending connection context
0809 15:36:07.032884 (+ 71us) client_negotiation.cc:240] Negotiation successful
0809 15:36:07.032957 (+ 73us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":988,"mutex_wait_us":41,"thread_start_us":86,"threads_started":1}
I20260809 15:36:07.035717 11004 tablet_service.cc:1467] Processing CreateTablet for tablet 0c0bd2ac825d4bc986f2e49979ca750d (DEFAULT_TABLE table=test-table [id=3c33bc37222749b9a69d258f00fb60f0]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 15:36:07.035705 11306 tablet_service.cc:1467] Processing CreateTablet for tablet 0c0bd2ac825d4bc986f2e49979ca750d (DEFAULT_TABLE table=test-table [id=3c33bc37222749b9a69d258f00fb60f0]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 15:36:07.036824 11004 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 0c0bd2ac825d4bc986f2e49979ca750d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 15:36:07.036824 11306 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 0c0bd2ac825d4bc986f2e49979ca750d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 15:36:07.036590 11155 tablet_service.cc:1467] Processing CreateTablet for tablet 0c0bd2ac825d4bc986f2e49979ca750d (DEFAULT_TABLE table=test-table [id=3c33bc37222749b9a69d258f00fb60f0]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 15:36:07.037433 11155 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 0c0bd2ac825d4bc986f2e49979ca750d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 15:36:07.044440 11416 tablet_bootstrap.cc:492] T 0c0bd2ac825d4bc986f2e49979ca750d P 7f9e63d41b3e46d9a25fb76d9463f92c: Bootstrap starting.
I20260809 15:36:07.045229 11417 tablet_bootstrap.cc:492] T 0c0bd2ac825d4bc986f2e49979ca750d P 4d0727bc39574ad3b523c62e60bb1615: Bootstrap starting.
I20260809 15:36:07.046756 11416 tablet_bootstrap.cc:654] T 0c0bd2ac825d4bc986f2e49979ca750d P 7f9e63d41b3e46d9a25fb76d9463f92c: Neither blocks nor log segments found. Creating new log.
I20260809 15:36:07.047593 11417 tablet_bootstrap.cc:654] T 0c0bd2ac825d4bc986f2e49979ca750d P 4d0727bc39574ad3b523c62e60bb1615: Neither blocks nor log segments found. Creating new log.
I20260809 15:36:07.047760 11416 log.cc:824] T 0c0bd2ac825d4bc986f2e49979ca750d P 7f9e63d41b3e46d9a25fb76d9463f92c: Log is configured to *not* fsync() on all Append() calls
I20260809 15:36:07.048491 11417 log.cc:824] T 0c0bd2ac825d4bc986f2e49979ca750d P 4d0727bc39574ad3b523c62e60bb1615: Log is configured to *not* fsync() on all Append() calls
I20260809 15:36:07.048561 11418 tablet_bootstrap.cc:492] T 0c0bd2ac825d4bc986f2e49979ca750d P 5cbab4c3b10e42448c1d2d3cd71ea8be: Bootstrap starting.
I20260809 15:36:07.050727 11418 tablet_bootstrap.cc:654] T 0c0bd2ac825d4bc986f2e49979ca750d P 5cbab4c3b10e42448c1d2d3cd71ea8be: Neither blocks nor log segments found. Creating new log.
I20260809 15:36:07.051357 11417 tablet_bootstrap.cc:492] T 0c0bd2ac825d4bc986f2e49979ca750d P 4d0727bc39574ad3b523c62e60bb1615: No bootstrap required, opened a new log
I20260809 15:36:07.051549 11417 ts_tablet_manager.cc:1397] T 0c0bd2ac825d4bc986f2e49979ca750d P 4d0727bc39574ad3b523c62e60bb1615: Time spent bootstrapping tablet: real 0.006s user 0.005s sys 0.000s
I20260809 15:36:07.052311 11416 tablet_bootstrap.cc:492] T 0c0bd2ac825d4bc986f2e49979ca750d P 7f9e63d41b3e46d9a25fb76d9463f92c: No bootstrap required, opened a new log
I20260809 15:36:07.052524 11416 ts_tablet_manager.cc:1397] T 0c0bd2ac825d4bc986f2e49979ca750d P 7f9e63d41b3e46d9a25fb76d9463f92c: Time spent bootstrapping tablet: real 0.008s user 0.001s sys 0.003s
I20260809 15:36:07.055042 11417 raft_consensus.cc:348] T 0c0bd2ac825d4bc986f2e49979ca750d P 4d0727bc39574ad3b523c62e60bb1615 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4d0727bc39574ad3b523c62e60bb1615" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 36983 } } peers { permanent_uuid: "5cbab4c3b10e42448c1d2d3cd71ea8be" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 44753 } } peers { permanent_uuid: "7f9e63d41b3e46d9a25fb76d9463f92c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 36959 } }
I20260809 15:36:07.055261 11417 raft_consensus.cc:374] T 0c0bd2ac825d4bc986f2e49979ca750d P 4d0727bc39574ad3b523c62e60bb1615 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 15:36:07.055352 11417 raft_consensus.cc:729] T 0c0bd2ac825d4bc986f2e49979ca750d P 4d0727bc39574ad3b523c62e60bb1615 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4d0727bc39574ad3b523c62e60bb1615, State: Initialized, Role: FOLLOWER
I20260809 15:36:07.055821 11418 log.cc:824] T 0c0bd2ac825d4bc986f2e49979ca750d P 5cbab4c3b10e42448c1d2d3cd71ea8be: Log is configured to *not* fsync() on all Append() calls
I20260809 15:36:07.055809 11417 consensus_queue.cc:260] T 0c0bd2ac825d4bc986f2e49979ca750d P 4d0727bc39574ad3b523c62e60bb1615 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4d0727bc39574ad3b523c62e60bb1615" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 36983 } } peers { permanent_uuid: "5cbab4c3b10e42448c1d2d3cd71ea8be" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 44753 } } peers { permanent_uuid: "7f9e63d41b3e46d9a25fb76d9463f92c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 36959 } }
I20260809 15:36:07.056285 11416 raft_consensus.cc:348] T 0c0bd2ac825d4bc986f2e49979ca750d P 7f9e63d41b3e46d9a25fb76d9463f92c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4d0727bc39574ad3b523c62e60bb1615" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 36983 } } peers { permanent_uuid: "5cbab4c3b10e42448c1d2d3cd71ea8be" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 44753 } } peers { permanent_uuid: "7f9e63d41b3e46d9a25fb76d9463f92c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 36959 } }
I20260809 15:36:07.056553 11379 heartbeater.cc:503] Master 127.3.89.254:46611 was elected leader, sending a full tablet report...
I20260809 15:36:07.056561 11416 raft_consensus.cc:374] T 0c0bd2ac825d4bc986f2e49979ca750d P 7f9e63d41b3e46d9a25fb76d9463f92c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 15:36:07.056623 11416 raft_consensus.cc:729] T 0c0bd2ac825d4bc986f2e49979ca750d P 7f9e63d41b3e46d9a25fb76d9463f92c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7f9e63d41b3e46d9a25fb76d9463f92c, State: Initialized, Role: FOLLOWER
I20260809 15:36:07.057030 11416 consensus_queue.cc:260] T 0c0bd2ac825d4bc986f2e49979ca750d P 7f9e63d41b3e46d9a25fb76d9463f92c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4d0727bc39574ad3b523c62e60bb1615" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 36983 } } peers { permanent_uuid: "5cbab4c3b10e42448c1d2d3cd71ea8be" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 44753 } } peers { permanent_uuid: "7f9e63d41b3e46d9a25fb76d9463f92c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 36959 } }
I20260809 15:36:07.058064 11416 ts_tablet_manager.cc:1428] T 0c0bd2ac825d4bc986f2e49979ca750d P 7f9e63d41b3e46d9a25fb76d9463f92c: Time spent starting tablet: real 0.005s user 0.004s sys 0.002s
I20260809 15:36:07.058357 11231 heartbeater.cc:503] Master 127.3.89.254:46611 was elected leader, sending a full tablet report...
I20260809 15:36:07.059916 11417 ts_tablet_manager.cc:1428] T 0c0bd2ac825d4bc986f2e49979ca750d P 4d0727bc39574ad3b523c62e60bb1615: Time spent starting tablet: real 0.008s user 0.005s sys 0.000s
I20260809 15:36:07.063977 11418 tablet_bootstrap.cc:492] T 0c0bd2ac825d4bc986f2e49979ca750d P 5cbab4c3b10e42448c1d2d3cd71ea8be: No bootstrap required, opened a new log
I20260809 15:36:07.064162 11418 ts_tablet_manager.cc:1397] T 0c0bd2ac825d4bc986f2e49979ca750d P 5cbab4c3b10e42448c1d2d3cd71ea8be: Time spent bootstrapping tablet: real 0.016s user 0.001s sys 0.003s
I20260809 15:36:07.067073 11418 raft_consensus.cc:348] T 0c0bd2ac825d4bc986f2e49979ca750d P 5cbab4c3b10e42448c1d2d3cd71ea8be [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4d0727bc39574ad3b523c62e60bb1615" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 36983 } } peers { permanent_uuid: "5cbab4c3b10e42448c1d2d3cd71ea8be" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 44753 } } peers { permanent_uuid: "7f9e63d41b3e46d9a25fb76d9463f92c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 36959 } }
I20260809 15:36:07.067293 11418 raft_consensus.cc:374] T 0c0bd2ac825d4bc986f2e49979ca750d P 5cbab4c3b10e42448c1d2d3cd71ea8be [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 15:36:07.067327 11418 raft_consensus.cc:729] T 0c0bd2ac825d4bc986f2e49979ca750d P 5cbab4c3b10e42448c1d2d3cd71ea8be [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5cbab4c3b10e42448c1d2d3cd71ea8be, State: Initialized, Role: FOLLOWER
I20260809 15:36:07.067708 11418 consensus_queue.cc:260] T 0c0bd2ac825d4bc986f2e49979ca750d P 5cbab4c3b10e42448c1d2d3cd71ea8be [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4d0727bc39574ad3b523c62e60bb1615" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 36983 } } peers { permanent_uuid: "5cbab4c3b10e42448c1d2d3cd71ea8be" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 44753 } } peers { permanent_uuid: "7f9e63d41b3e46d9a25fb76d9463f92c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 36959 } }
I20260809 15:36:07.068547 11418 ts_tablet_manager.cc:1428] T 0c0bd2ac825d4bc986f2e49979ca750d P 5cbab4c3b10e42448c1d2d3cd71ea8be: Time spent starting tablet: real 0.004s user 0.005s sys 0.000s
I20260809 15:36:07.068738 11079 heartbeater.cc:503] Master 127.3.89.254:46611 was elected leader, sending a full tablet report...
W20260809 15:36:07.226168 11381 tablet.cc:2369] T 0c0bd2ac825d4bc986f2e49979ca750d P 4d0727bc39574ad3b523c62e60bb1615: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260809 15:36:07.267318 11233 tablet.cc:2369] T 0c0bd2ac825d4bc986f2e49979ca750d P 7f9e63d41b3e46d9a25fb76d9463f92c: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260809 15:36:07.288148 11423 raft_consensus.cc:482] T 0c0bd2ac825d4bc986f2e49979ca750d P 7f9e63d41b3e46d9a25fb76d9463f92c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 15:36:07.288383 11423 raft_consensus.cc:504] T 0c0bd2ac825d4bc986f2e49979ca750d P 7f9e63d41b3e46d9a25fb76d9463f92c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4d0727bc39574ad3b523c62e60bb1615" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 36983 } } peers { permanent_uuid: "5cbab4c3b10e42448c1d2d3cd71ea8be" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 44753 } } peers { permanent_uuid: "7f9e63d41b3e46d9a25fb76d9463f92c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 36959 } }
I20260809 15:36:07.289565 11423 leader_election.cc:288] T 0c0bd2ac825d4bc986f2e49979ca750d P 7f9e63d41b3e46d9a25fb76d9463f92c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 4d0727bc39574ad3b523c62e60bb1615 (127.3.89.195:36983), 5cbab4c3b10e42448c1d2d3cd71ea8be (127.3.89.193:44753)
I20260809 15:36:07.295131 11411 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:36:07.289962 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.89.194:55997 (local address 127.3.89.195:36983)
0809 15:36:07.290211 (+ 249us) server_negotiation.cc:206] Beginning negotiation
0809 15:36:07.290214 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0809 15:36:07.290229 (+ 15us) server_negotiation.cc:407] Connection header received
0809 15:36:07.290411 (+ 182us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 15:36:07.290414 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 15:36:07.290464 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 15:36:07.290553 (+ 89us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 15:36:07.290984 (+ 431us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:36:07.291777 (+ 793us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:36:07.293637 (+ 1860us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:36:07.294266 (+ 629us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:36:07.294413 (+ 147us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:36:07.294467 (+ 54us) server_negotiation.cc:1035] Waiting for connection context
0809 15:36:07.294897 (+ 430us) server_negotiation.cc:299] Negotiation successful
0809 15:36:07.294993 (+ 96us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":154}
I20260809 15:36:07.295131 11413 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:36:07.289981 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.89.194:48473 (local address 127.3.89.193:44753)
0809 15:36:07.290138 (+ 157us) server_negotiation.cc:206] Beginning negotiation
0809 15:36:07.290142 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 15:36:07.290301 (+ 159us) server_negotiation.cc:407] Connection header received
0809 15:36:07.290402 (+ 101us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 15:36:07.290406 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 15:36:07.290464 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 15:36:07.290553 (+ 89us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 15:36:07.290984 (+ 431us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:36:07.291778 (+ 794us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:36:07.293637 (+ 1859us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:36:07.294289 (+ 652us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:36:07.294413 (+ 124us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:36:07.294467 (+ 54us) server_negotiation.cc:1035] Waiting for connection context
0809 15:36:07.294897 (+ 430us) server_negotiation.cc:299] Negotiation successful
0809 15:36:07.294993 (+ 96us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":66}
I20260809 15:36:07.295215 11425 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:36:07.289848 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.89.193:44753 (local address 127.3.89.194:48473)
0809 15:36:07.290214 (+ 366us) negotiation.cc:107] Waiting for socket to connect
0809 15:36:07.290227 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0809 15:36:07.290303 (+ 76us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 15:36:07.290579 (+ 276us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 15:36:07.290582 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 15:36:07.290616 (+ 34us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 15:36:07.290851 (+ 235us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:36:07.290856 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:36:07.291899 (+ 1043us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:36:07.291902 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:36:07.293489 (+ 1587us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:36:07.293502 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:36:07.294413 (+ 911us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:36:07.294417 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:36:07.294685 (+ 268us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:36:07.294782 (+ 97us) client_negotiation.cc:769] Sending connection context
0809 15:36:07.294862 (+ 80us) client_negotiation.cc:240] Negotiation successful
0809 15:36:07.295018 (+ 156us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":298,"mutex_wait_us":401,"thread_start_us":90,"threads_started":1}
I20260809 15:36:07.295619 11426 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:36:07.289848 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.89.195:36983 (local address 127.3.89.194:55997)
0809 15:36:07.290144 (+ 296us) negotiation.cc:107] Waiting for socket to connect
0809 15:36:07.290161 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0809 15:36:07.290289 (+ 128us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 15:36:07.290579 (+ 290us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 15:36:07.290582 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 15:36:07.290629 (+ 47us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 15:36:07.290848 (+ 219us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:36:07.290855 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:36:07.291899 (+ 1044us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:36:07.291902 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:36:07.293490 (+ 1588us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:36:07.293502 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:36:07.294413 (+ 911us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:36:07.294417 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:36:07.294682 (+ 265us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:36:07.294782 (+ 100us) client_negotiation.cc:769] Sending connection context
0809 15:36:07.294862 (+ 80us) client_negotiation.cc:240] Negotiation successful
0809 15:36:07.295017 (+ 155us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":213,"mutex_wait_us":362,"spinlock_wait_cycles":52864,"thread_start_us":103,"threads_started":1}
I20260809 15:36:07.295929 11029 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0c0bd2ac825d4bc986f2e49979ca750d" candidate_uuid: "7f9e63d41b3e46d9a25fb76d9463f92c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5cbab4c3b10e42448c1d2d3cd71ea8be" is_pre_election: true
I20260809 15:36:07.296298 11029 raft_consensus.cc:2454] T 0c0bd2ac825d4bc986f2e49979ca750d P 5cbab4c3b10e42448c1d2d3cd71ea8be [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7f9e63d41b3e46d9a25fb76d9463f92c in term 0.
I20260809 15:36:07.296355 11329 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0c0bd2ac825d4bc986f2e49979ca750d" candidate_uuid: "7f9e63d41b3e46d9a25fb76d9463f92c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4d0727bc39574ad3b523c62e60bb1615" is_pre_election: true
I20260809 15:36:07.296649 11329 raft_consensus.cc:2454] T 0c0bd2ac825d4bc986f2e49979ca750d P 4d0727bc39574ad3b523c62e60bb1615 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7f9e63d41b3e46d9a25fb76d9463f92c in term 0.
I20260809 15:36:07.296790 11103 leader_election.cc:302] T 0c0bd2ac825d4bc986f2e49979ca750d P 7f9e63d41b3e46d9a25fb76d9463f92c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5cbab4c3b10e42448c1d2d3cd71ea8be, 7f9e63d41b3e46d9a25fb76d9463f92c; no voters:
I20260809 15:36:07.297106 11423 raft_consensus.cc:2781] T 0c0bd2ac825d4bc986f2e49979ca750d P 7f9e63d41b3e46d9a25fb76d9463f92c [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260809 15:36:07.297191 11423 raft_consensus.cc:482] T 0c0bd2ac825d4bc986f2e49979ca750d P 7f9e63d41b3e46d9a25fb76d9463f92c [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 15:36:07.297241 11423 raft_consensus.cc:3037] T 0c0bd2ac825d4bc986f2e49979ca750d P 7f9e63d41b3e46d9a25fb76d9463f92c [term 0 FOLLOWER]: Advancing to term 1
I20260809 15:36:07.298271 11423 raft_consensus.cc:504] T 0c0bd2ac825d4bc986f2e49979ca750d P 7f9e63d41b3e46d9a25fb76d9463f92c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4d0727bc39574ad3b523c62e60bb1615" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 36983 } } peers { permanent_uuid: "5cbab4c3b10e42448c1d2d3cd71ea8be" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 44753 } } peers { permanent_uuid: "7f9e63d41b3e46d9a25fb76d9463f92c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 36959 } }
I20260809 15:36:07.298672 11423 leader_election.cc:288] T 0c0bd2ac825d4bc986f2e49979ca750d P 7f9e63d41b3e46d9a25fb76d9463f92c [CANDIDATE]: Term 1 election: Requested vote from peers 4d0727bc39574ad3b523c62e60bb1615 (127.3.89.195:36983), 5cbab4c3b10e42448c1d2d3cd71ea8be (127.3.89.193:44753)
I20260809 15:36:07.299027 11329 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0c0bd2ac825d4bc986f2e49979ca750d" candidate_uuid: "7f9e63d41b3e46d9a25fb76d9463f92c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4d0727bc39574ad3b523c62e60bb1615"
I20260809 15:36:07.299048 11029 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0c0bd2ac825d4bc986f2e49979ca750d" candidate_uuid: "7f9e63d41b3e46d9a25fb76d9463f92c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5cbab4c3b10e42448c1d2d3cd71ea8be"
I20260809 15:36:07.299139 11329 raft_consensus.cc:3037] T 0c0bd2ac825d4bc986f2e49979ca750d P 4d0727bc39574ad3b523c62e60bb1615 [term 0 FOLLOWER]: Advancing to term 1
I20260809 15:36:07.299175 11029 raft_consensus.cc:3037] T 0c0bd2ac825d4bc986f2e49979ca750d P 5cbab4c3b10e42448c1d2d3cd71ea8be [term 0 FOLLOWER]: Advancing to term 1
I20260809 15:36:07.300194 11329 raft_consensus.cc:2454] T 0c0bd2ac825d4bc986f2e49979ca750d P 4d0727bc39574ad3b523c62e60bb1615 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7f9e63d41b3e46d9a25fb76d9463f92c in term 1.
I20260809 15:36:07.300206 11029 raft_consensus.cc:2454] T 0c0bd2ac825d4bc986f2e49979ca750d P 5cbab4c3b10e42448c1d2d3cd71ea8be [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7f9e63d41b3e46d9a25fb76d9463f92c in term 1.
I20260809 15:36:07.300478 11107 leader_election.cc:302] T 0c0bd2ac825d4bc986f2e49979ca750d P 7f9e63d41b3e46d9a25fb76d9463f92c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4d0727bc39574ad3b523c62e60bb1615, 7f9e63d41b3e46d9a25fb76d9463f92c; no voters:
I20260809 15:36:07.300714 11423 raft_consensus.cc:2781] T 0c0bd2ac825d4bc986f2e49979ca750d P 7f9e63d41b3e46d9a25fb76d9463f92c [term 1 FOLLOWER]: Leader election won for term 1
I20260809 15:36:07.300907 11423 raft_consensus.cc:686] T 0c0bd2ac825d4bc986f2e49979ca750d P 7f9e63d41b3e46d9a25fb76d9463f92c [term 1 LEADER]: Becoming Leader. State: Replica: 7f9e63d41b3e46d9a25fb76d9463f92c, State: Running, Role: LEADER
I20260809 15:36:07.301268 11423 consensus_queue.cc:237] T 0c0bd2ac825d4bc986f2e49979ca750d P 7f9e63d41b3e46d9a25fb76d9463f92c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4d0727bc39574ad3b523c62e60bb1615" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 36983 } } peers { permanent_uuid: "5cbab4c3b10e42448c1d2d3cd71ea8be" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 44753 } } peers { permanent_uuid: "7f9e63d41b3e46d9a25fb76d9463f92c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 36959 } }
W20260809 15:36:07.304570 11081 tablet.cc:2369] T 0c0bd2ac825d4bc986f2e49979ca750d P 5cbab4c3b10e42448c1d2d3cd71ea8be: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260809 15:36:07.305172 10012 catalog_manager.cc:5495] T 0c0bd2ac825d4bc986f2e49979ca750d P 7f9e63d41b3e46d9a25fb76d9463f92c reported cstate change: term changed from 0 to 1, leader changed from <none> to 7f9e63d41b3e46d9a25fb76d9463f92c (127.3.89.194). New cstate: current_term: 1 leader_uuid: "7f9e63d41b3e46d9a25fb76d9463f92c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4d0727bc39574ad3b523c62e60bb1615" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 36983 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5cbab4c3b10e42448c1d2d3cd71ea8be" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 44753 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7f9e63d41b3e46d9a25fb76d9463f92c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 36959 } health_report { overall_health: HEALTHY } } }
I20260809 15:36:07.319628 11415 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:36:07.315798 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37022 (local address 127.3.89.194:36959)
0809 15:36:07.315925 (+ 127us) server_negotiation.cc:206] Beginning negotiation
0809 15:36:07.315930 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 15:36:07.315978 (+ 48us) server_negotiation.cc:407] Connection header received
0809 15:36:07.316162 (+ 184us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 15:36:07.316166 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 15:36:07.316226 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 15:36:07.316335 (+ 109us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 15:36:07.316619 (+ 284us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:36:07.317190 (+ 571us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:36:07.318047 (+ 857us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:36:07.318334 (+ 287us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:36:07.318437 (+ 103us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:36:07.318753 (+ 316us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 15:36:07.319226 (+ 473us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 15:36:07.319358 (+ 132us) server_negotiation.cc:1035] Waiting for connection context
0809 15:36:07.319425 (+ 67us) server_negotiation.cc:299] Negotiation successful
0809 15:36:07.319487 (+ 62us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":43}
I20260809 15:36:07.325163 11029 raft_consensus.cc:1261] T 0c0bd2ac825d4bc986f2e49979ca750d P 5cbab4c3b10e42448c1d2d3cd71ea8be [term 1 FOLLOWER]: Refusing update from remote peer 7f9e63d41b3e46d9a25fb76d9463f92c: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 15:36:07.325259 11329 raft_consensus.cc:1261] T 0c0bd2ac825d4bc986f2e49979ca750d P 4d0727bc39574ad3b523c62e60bb1615 [term 1 FOLLOWER]: Refusing update from remote peer 7f9e63d41b3e46d9a25fb76d9463f92c: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 15:36:07.326030 11423 consensus_queue.cc:1035] T 0c0bd2ac825d4bc986f2e49979ca750d P 7f9e63d41b3e46d9a25fb76d9463f92c [LEADER]: Connected to new peer: Peer: permanent_uuid: "5cbab4c3b10e42448c1d2d3cd71ea8be" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 44753 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 15:36:07.326907 11423 consensus_queue.cc:1035] T 0c0bd2ac825d4bc986f2e49979ca750d P 7f9e63d41b3e46d9a25fb76d9463f92c [LEADER]: Connected to new peer: Peer: permanent_uuid: "4d0727bc39574ad3b523c62e60bb1615" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 36983 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 15:36:07.337329 11432 mvcc.cc:205] Tried to move back new op lower bound from 7316642886956879872 to 7316642886868660224. Current Snapshot: MvccSnapshot[applied={T|T < 7316642886956879872}]
I20260809 15:36:07.337538 11433 mvcc.cc:205] Tried to move back new op lower bound from 7316642886956879872 to 7316642886868660224. Current Snapshot: MvccSnapshot[applied={T|T < 7316642886956879872}]
I20260809 15:36:07.347393 10012 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:53188:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 15:36:07.347537 10012 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:53188:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 15:36:07.351570 10012 catalog_manager.cc:5782] T 00000000000000000000000000000000 P b7ef4cef863642f19bd9f01ae2792fe3: Sending DeleteTablet for 3 replicas of tablet 0c0bd2ac825d4bc986f2e49979ca750d
I20260809 15:36:07.352376 11004 tablet_service.cc:1514] Processing DeleteTablet for tablet 0c0bd2ac825d4bc986f2e49979ca750d with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 15:36:07 UTC) from {username='oryx', principal='oryx/127.3.89.254@KRBTEST.COM'} at 127.0.0.1:51608
I20260809 15:36:07.352537 11306 tablet_service.cc:1514] Processing DeleteTablet for tablet 0c0bd2ac825d4bc986f2e49979ca750d with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 15:36:07 UTC) from {username='oryx', principal='oryx/127.3.89.254@KRBTEST.COM'} at 127.0.0.1:39280
I20260809 15:36:07.352936 11442 tablet_replica.cc:330] T 0c0bd2ac825d4bc986f2e49979ca750d P 4d0727bc39574ad3b523c62e60bb1615: stopping tablet replica
I20260809 15:36:07.353256 11442 raft_consensus.cc:2229] T 0c0bd2ac825d4bc986f2e49979ca750d P 4d0727bc39574ad3b523c62e60bb1615 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 15:36:07.353575 11442 raft_consensus.cc:2258] T 0c0bd2ac825d4bc986f2e49979ca750d P 4d0727bc39574ad3b523c62e60bb1615 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 15:36:07.353820 11155 tablet_service.cc:1514] Processing DeleteTablet for tablet 0c0bd2ac825d4bc986f2e49979ca750d with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 15:36:07 UTC) from {username='oryx', principal='oryx/127.3.89.254@KRBTEST.COM'} at 127.0.0.1:37016
I20260809 15:36:07.355163 11444 tablet_replica.cc:330] T 0c0bd2ac825d4bc986f2e49979ca750d P 7f9e63d41b3e46d9a25fb76d9463f92c: stopping tablet replica
I20260809 15:36:07.355219 11442 ts_tablet_manager.cc:1905] T 0c0bd2ac825d4bc986f2e49979ca750d P 4d0727bc39574ad3b523c62e60bb1615: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 15:36:07.355477 11444 raft_consensus.cc:2229] T 0c0bd2ac825d4bc986f2e49979ca750d P 7f9e63d41b3e46d9a25fb76d9463f92c [term 1 LEADER]: Raft consensus shutting down.
I20260809 15:36:07.355811 3431 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskm314YR/build/debug/bin/kudu with pid 9978
I20260809 15:36:07.355911 11444 raft_consensus.cc:2258] T 0c0bd2ac825d4bc986f2e49979ca750d P 7f9e63d41b3e46d9a25fb76d9463f92c [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 15:36:07.358731 11442 ts_tablet_manager.cc:1918] T 0c0bd2ac825d4bc986f2e49979ca750d P 4d0727bc39574ad3b523c62e60bb1615: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 15:36:07.358843 11442 log.cc:1196] T 0c0bd2ac825d4bc986f2e49979ca750d P 4d0727bc39574ad3b523c62e60bb1615: Deleting WAL directory at /tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786289732823074-3431-0/minicluster-data/ts-2/wal/wals/0c0bd2ac825d4bc986f2e49979ca750d
I20260809 15:36:07.359215 11442 ts_tablet_manager.cc:1939] T 0c0bd2ac825d4bc986f2e49979ca750d P 4d0727bc39574ad3b523c62e60bb1615: Deleting consensus metadata
I20260809 15:36:07.359633 11444 ts_tablet_manager.cc:1905] T 0c0bd2ac825d4bc986f2e49979ca750d P 7f9e63d41b3e46d9a25fb76d9463f92c: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 15:36:07.361773 11444 ts_tablet_manager.cc:1918] T 0c0bd2ac825d4bc986f2e49979ca750d P 7f9e63d41b3e46d9a25fb76d9463f92c: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 15:36:07.361893 11444 log.cc:1196] T 0c0bd2ac825d4bc986f2e49979ca750d P 7f9e63d41b3e46d9a25fb76d9463f92c: Deleting WAL directory at /tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786289732823074-3431-0/minicluster-data/ts-1/wal/wals/0c0bd2ac825d4bc986f2e49979ca750d
I20260809 15:36:07.362174 11444 ts_tablet_manager.cc:1939] T 0c0bd2ac825d4bc986f2e49979ca750d P 7f9e63d41b3e46d9a25fb76d9463f92c: Deleting consensus metadata
W20260809 15:36:07.366948 11107 connection.cc:535] server connection from 127.0.0.1:37016 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
W20260809 15:36:07.367311 11253 connection.cc:535] server connection from 127.0.0.1:39280 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
I20260809 15:36:07.367471 11441 tablet_replica.cc:330] T 0c0bd2ac825d4bc986f2e49979ca750d P 5cbab4c3b10e42448c1d2d3cd71ea8be: stopping tablet replica
I20260809 15:36:07.367816 11441 raft_consensus.cc:2229] T 0c0bd2ac825d4bc986f2e49979ca750d P 5cbab4c3b10e42448c1d2d3cd71ea8be [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 15:36:07.368165 11441 raft_consensus.cc:2258] T 0c0bd2ac825d4bc986f2e49979ca750d P 5cbab4c3b10e42448c1d2d3cd71ea8be [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 15:36:07.370594 11441 ts_tablet_manager.cc:1905] T 0c0bd2ac825d4bc986f2e49979ca750d P 5cbab4c3b10e42448c1d2d3cd71ea8be: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 15:36:07.373338 11441 ts_tablet_manager.cc:1918] T 0c0bd2ac825d4bc986f2e49979ca750d P 5cbab4c3b10e42448c1d2d3cd71ea8be: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 15:36:07.373447 11441 log.cc:1196] T 0c0bd2ac825d4bc986f2e49979ca750d P 5cbab4c3b10e42448c1d2d3cd71ea8be: Deleting WAL directory at /tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786289732823074-3431-0/minicluster-data/ts-0/wal/wals/0c0bd2ac825d4bc986f2e49979ca750d
I20260809 15:36:07.373849 11441 ts_tablet_manager.cc:1939] T 0c0bd2ac825d4bc986f2e49979ca750d P 5cbab4c3b10e42448c1d2d3cd71ea8be: Deleting consensus metadata
W20260809 15:36:07.375075 10953 connection.cc:441] server connection from 127.0.0.1:51608 torn down before Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:51608 (request call id 1) could send its response
I20260809 15:36:07.378331 11391 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:36:07.354208 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36526 (local address 127.3.89.253:42075)
0809 15:36:07.354327 (+ 119us) server_negotiation.cc:206] Beginning negotiation
0809 15:36:07.354331 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 15:36:07.367372 (+ 13041us) server_negotiation.cc:407] Connection header received
0809 15:36:07.367667 (+ 295us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 15:36:07.367670 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 15:36:07.367723 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 15:36:07.367776 (+ 53us) server_negotiation.cc:226] Negotiated authn=SASL
0809 15:36:07.369071 (+ 1295us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:36:07.369804 (+ 733us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:36:07.370676 (+ 872us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:36:07.371101 (+ 425us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:36:07.371205 (+ 104us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:36:07.373205 (+ 2000us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 15:36:07.373212 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 15:36:07.373218 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 15:36:07.373250 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0809 15:36:07.375328 (+ 2078us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 15:36:07.376092 (+ 764us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 15:36:07.376099 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 15:36:07.376104 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0809 15:36:07.376200 (+ 96us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 15:36:07.376531 (+ 331us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 15:36:07.376534 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 15:36:07.376536 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 15:36:07.376770 (+ 234us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 15:36:07.376900 (+ 130us) server_negotiation.cc:1035] Waiting for connection context
0809 15:36:07.377995 (+ 1095us) server_negotiation.cc:299] Negotiation successful
0809 15:36:07.378128 (+ 133us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":42}
I20260809 15:36:07.383194 10809 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:36:07.372053 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36532 (local address 127.3.89.253:42075)
0809 15:36:07.372168 (+ 115us) server_negotiation.cc:206] Beginning negotiation
0809 15:36:07.372171 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0809 15:36:07.374165 (+ 1994us) server_negotiation.cc:407] Connection header received
0809 15:36:07.374318 (+ 153us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 15:36:07.374321 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 15:36:07.374363 (+ 42us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 15:36:07.374410 (+ 47us) server_negotiation.cc:226] Negotiated authn=SASL
0809 15:36:07.375256 (+ 846us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:36:07.376031 (+ 775us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:36:07.377235 (+ 1204us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:36:07.377557 (+ 322us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:36:07.377624 (+ 67us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:36:07.379340 (+ 1716us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 15:36:07.379344 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 15:36:07.379347 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 15:36:07.379371 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0809 15:36:07.381252 (+ 1881us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 15:36:07.381857 (+ 605us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 15:36:07.381862 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 15:36:07.381864 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 15:36:07.381945 (+ 81us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 15:36:07.382242 (+ 297us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 15:36:07.382245 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 15:36:07.382247 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 15:36:07.382433 (+ 186us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 15:36:07.382530 (+ 97us) server_negotiation.cc:1035] Waiting for connection context
0809 15:36:07.382909 (+ 379us) server_negotiation.cc:299] Negotiation successful
0809 15:36:07.383042 (+ 133us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":35}
I20260809 15:36:07.385676 11394 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:36:07.373519 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33372 (local address 127.3.89.252:44407)
0809 15:36:07.373815 (+ 296us) server_negotiation.cc:206] Beginning negotiation
0809 15:36:07.373819 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 15:36:07.375514 (+ 1695us) server_negotiation.cc:407] Connection header received
0809 15:36:07.375704 (+ 190us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 15:36:07.375707 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 15:36:07.375768 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 15:36:07.375848 (+ 80us) server_negotiation.cc:226] Negotiated authn=SASL
0809 15:36:07.376743 (+ 895us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:36:07.377506 (+ 763us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:36:07.378883 (+ 1377us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:36:07.379240 (+ 357us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:36:07.379334 (+ 94us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:36:07.381881 (+ 2547us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 15:36:07.381884 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 15:36:07.381887 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 15:36:07.381911 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0809 15:36:07.383850 (+ 1939us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 15:36:07.384324 (+ 474us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 15:36:07.384328 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 15:36:07.384329 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 15:36:07.384414 (+ 85us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 15:36:07.384706 (+ 292us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 15:36:07.384710 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 15:36:07.384712 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 15:36:07.384916 (+ 204us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 15:36:07.385011 (+ 95us) server_negotiation.cc:1035] Waiting for connection context
0809 15:36:07.385382 (+ 371us) server_negotiation.cc:299] Negotiation successful
0809 15:36:07.385502 (+ 120us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":198}
I20260809 15:36:07.387830 11396 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:36:07.354079 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33370 (local address 127.3.89.252:44407)
0809 15:36:07.354204 (+ 125us) server_negotiation.cc:206] Beginning negotiation
0809 15:36:07.354209 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 15:36:07.377684 (+ 23475us) server_negotiation.cc:407] Connection header received
0809 15:36:07.377720 (+ 36us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 15:36:07.377722 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 15:36:07.377769 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 15:36:07.377850 (+ 81us) server_negotiation.cc:226] Negotiated authn=SASL
0809 15:36:07.379931 (+ 2081us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:36:07.380597 (+ 666us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:36:07.381829 (+ 1232us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:36:07.382161 (+ 332us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:36:07.382269 (+ 108us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:36:07.384078 (+ 1809us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 15:36:07.384082 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 15:36:07.384084 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 15:36:07.384108 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0809 15:36:07.385853 (+ 1745us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 15:36:07.386343 (+ 490us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 15:36:07.386347 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 15:36:07.386348 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 15:36:07.386465 (+ 117us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 15:36:07.386760 (+ 295us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 15:36:07.386763 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 15:36:07.386765 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 15:36:07.386926 (+ 161us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 15:36:07.387335 (+ 409us) server_negotiation.cc:1035] Waiting for connection context
0809 15:36:07.387560 (+ 225us) server_negotiation.cc:299] Negotiation successful
0809 15:36:07.387676 (+ 116us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":50}
I20260809 15:36:08.340544 11453 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:36:08.340011 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.89.254:46611 (local address 127.3.89.194:44973)
0809 15:36:08.340309 (+ 298us) negotiation.cc:107] Waiting for socket to connect
0809 15:36:08.340412 (+ 103us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.89.254:46611: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":207,"thread_start_us":89,"threads_started":1}
W20260809 15:36:08.340886 11231 heartbeater.cc:644] Failed to heartbeat to 127.3.89.254:46611 (0 consecutive failures): Network error: Failed to send heartbeat to master: Client connection negotiation failed: client connection to 127.3.89.254:46611: connect: Connection refused (error 111)
I20260809 15:36:08.343272 11454 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:36:08.342854 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.89.254:46611 (local address 127.3.89.193:56527)
0809 15:36:08.343120 (+ 266us) negotiation.cc:107] Waiting for socket to connect
0809 15:36:08.343195 (+ 75us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.89.254:46611: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":202,"thread_start_us":66,"threads_started":1}
W20260809 15:36:08.343686 11079 heartbeater.cc:644] Failed to heartbeat to 127.3.89.254:46611 (0 consecutive failures): Network error: Failed to send heartbeat to master: Client connection negotiation failed: client connection to 127.3.89.254:46611: connect: Connection refused (error 111)
I20260809 15:36:08.343887 11456 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:36:08.343465 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.89.254:46611 (local address 127.3.89.195:59235)
0809 15:36:08.343704 (+ 239us) negotiation.cc:107] Waiting for socket to connect
0809 15:36:08.343803 (+ 99us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.89.254:46611: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":163,"thread_start_us":69,"threads_started":1}
W20260809 15:36:08.344118 11379 heartbeater.cc:644] Failed to heartbeat to 127.3.89.254:46611 (0 consecutive failures): Network error: Failed to send heartbeat to master: Client connection negotiation failed: client connection to 127.3.89.254:46611: connect: Connection refused (error 111)
I20260809 15:36:08.852352 11457 raft_consensus.cc:482] T 00000000000000000000000000000000 P d36c8ca8de404275bc63795e17cb197e [term 1 FOLLOWER]: Starting pre-election (detected failure of leader b7ef4cef863642f19bd9f01ae2792fe3)
I20260809 15:36:08.852489 11457 raft_consensus.cc:504] T 00000000000000000000000000000000 P d36c8ca8de404275bc63795e17cb197e [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "b7ef4cef863642f19bd9f01ae2792fe3" member_type: VOTER last_known_addr { host: "127.3.89.254" port: 46611 } } peers { permanent_uuid: "d36c8ca8de404275bc63795e17cb197e" member_type: VOTER last_known_addr { host: "127.3.89.253" port: 42075 } attrs { promote: false } } peers { permanent_uuid: "4bb903b269ca47afa0e1d4334950ab94" member_type: VOTER last_known_addr { host: "127.3.89.252" port: 44407 } attrs { promote: false } }
I20260809 15:36:08.852923 11457 leader_election.cc:288] T 00000000000000000000000000000000 P d36c8ca8de404275bc63795e17cb197e [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers b7ef4cef863642f19bd9f01ae2792fe3 (127.3.89.254:46611), 4bb903b269ca47afa0e1d4334950ab94 (127.3.89.252:44407)
I20260809 15:36:08.853575 11458 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:36:08.853095 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.89.254:46611 (local address 127.0.0.1:53376)
0809 15:36:08.853391 (+ 296us) negotiation.cc:107] Waiting for socket to connect
0809 15:36:08.853493 (+ 102us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.89.254:46611: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":211,"thread_start_us":72,"threads_started":1}
W20260809 15:36:08.853837 10661 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.3.89.254:46611: connect: Connection refused (error 111)
I20260809 15:36:08.854655 11458 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:36:08.854463 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.89.254:46611 (local address 127.0.0.1:53388)
0809 15:36:08.854558 (+ 95us) negotiation.cc:107] Waiting for socket to connect
0809 15:36:08.854602 (+ 44us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.89.254:46611: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":42}
W20260809 15:36:08.854988 10661 leader_election.cc:334] T 00000000000000000000000000000000 P d36c8ca8de404275bc63795e17cb197e [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer b7ef4cef863642f19bd9f01ae2792fe3 (127.3.89.254:46611): Network error: Client connection negotiation failed: client connection to 127.3.89.254:46611: connect: Connection refused (error 111)
I20260809 15:36:08.855361 11462 raft_consensus.cc:482] T 00000000000000000000000000000000 P 4bb903b269ca47afa0e1d4334950ab94 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader b7ef4cef863642f19bd9f01ae2792fe3)
I20260809 15:36:08.855526 11462 raft_consensus.cc:504] T 00000000000000000000000000000000 P 4bb903b269ca47afa0e1d4334950ab94 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "b7ef4cef863642f19bd9f01ae2792fe3" member_type: VOTER last_known_addr { host: "127.3.89.254" port: 46611 } } peers { permanent_uuid: "d36c8ca8de404275bc63795e17cb197e" member_type: VOTER last_known_addr { host: "127.3.89.253" port: 42075 } attrs { promote: false } } peers { permanent_uuid: "4bb903b269ca47afa0e1d4334950ab94" member_type: VOTER last_known_addr { host: "127.3.89.252" port: 44407 } attrs { promote: false } }
I20260809 15:36:08.856494 11462 leader_election.cc:288] T 00000000000000000000000000000000 P 4bb903b269ca47afa0e1d4334950ab94 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers b7ef4cef863642f19bd9f01ae2792fe3 (127.3.89.254:46611), d36c8ca8de404275bc63795e17cb197e (127.3.89.253:42075)
I20260809 15:36:08.857014 11463 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:36:08.856624 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.89.254:46611 (local address 127.0.0.1:53400)
0809 15:36:08.856849 (+ 225us) negotiation.cc:107] Waiting for socket to connect
0809 15:36:08.856929 (+ 80us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.89.254:46611: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":153,"thread_start_us":72,"threads_started":1}
W20260809 15:36:08.857309 10863 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.3.89.254:46611: connect: Connection refused (error 111)
I20260809 15:36:08.857719 11459 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:36:08.853095 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.89.252:44407 (local address 127.0.0.1:51710)
0809 15:36:08.853429 (+ 334us) negotiation.cc:107] Waiting for socket to connect
0809 15:36:08.853445 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0809 15:36:08.853544 (+ 99us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 15:36:08.853840 (+ 296us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 15:36:08.853842 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 15:36:08.853861 (+ 19us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 15:36:08.853998 (+ 137us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:36:08.854010 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:36:08.855053 (+ 1043us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:36:08.855056 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:36:08.856059 (+ 1003us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:36:08.856069 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:36:08.857301 (+ 1232us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:36:08.857305 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:36:08.857408 (+ 103us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:36:08.857437 (+ 29us) client_negotiation.cc:769] Sending connection context
0809 15:36:08.857518 (+ 81us) client_negotiation.cc:240] Negotiation successful
0809 15:36:08.857593 (+ 75us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":252,"mutex_wait_us":50,"thread_start_us":62,"threads_started":1}
I20260809 15:36:08.857749 11460 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:36:08.853303 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51710 (local address 127.3.89.252:44407)
0809 15:36:08.853542 (+ 239us) server_negotiation.cc:206] Beginning negotiation
0809 15:36:08.853547 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 15:36:08.853562 (+ 15us) server_negotiation.cc:407] Connection header received
0809 15:36:08.853640 (+ 78us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 15:36:08.853643 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 15:36:08.853716 (+ 73us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 15:36:08.853814 (+ 98us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 15:36:08.854123 (+ 309us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:36:08.854929 (+ 806us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:36:08.856182 (+ 1253us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:36:08.857090 (+ 908us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:36:08.857242 (+ 152us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:36:08.857310 (+ 68us) server_negotiation.cc:1035] Waiting for connection context
0809 15:36:08.857541 (+ 231us) server_negotiation.cc:299] Negotiation successful
0809 15:36:08.857614 (+ 73us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":167,"thread_start_us":73,"threads_started":1}
I20260809 15:36:08.858173 11463 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:36:08.857948 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.89.254:46611 (local address 127.0.0.1:53412)
0809 15:36:08.858055 (+ 107us) negotiation.cc:107] Waiting for socket to connect
0809 15:36:08.858114 (+ 59us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.89.254:46611: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":40}
W20260809 15:36:08.858323 10863 leader_election.cc:334] T 00000000000000000000000000000000 P 4bb903b269ca47afa0e1d4334950ab94 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer b7ef4cef863642f19bd9f01ae2792fe3 (127.3.89.254:46611): Network error: Client connection negotiation failed: client connection to 127.3.89.254:46611: connect: Connection refused (error 111)
I20260809 15:36:08.858377 10890 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "d36c8ca8de404275bc63795e17cb197e" candidate_term: 2 candidate_status { last_received { term: 1 index: 12 } } ignore_live_leader: false dest_uuid: "4bb903b269ca47afa0e1d4334950ab94" is_pre_election: true
I20260809 15:36:08.858605 10890 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 4bb903b269ca47afa0e1d4334950ab94 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d36c8ca8de404275bc63795e17cb197e in term 1.
I20260809 15:36:08.858947 10662 leader_election.cc:302] T 00000000000000000000000000000000 P d36c8ca8de404275bc63795e17cb197e [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4bb903b269ca47afa0e1d4334950ab94, d36c8ca8de404275bc63795e17cb197e; no voters: b7ef4cef863642f19bd9f01ae2792fe3
I20260809 15:36:08.859211 11457 raft_consensus.cc:2781] T 00000000000000000000000000000000 P d36c8ca8de404275bc63795e17cb197e [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260809 15:36:08.859277 11457 raft_consensus.cc:482] T 00000000000000000000000000000000 P d36c8ca8de404275bc63795e17cb197e [term 1 FOLLOWER]: Starting leader election (detected failure of leader b7ef4cef863642f19bd9f01ae2792fe3)
I20260809 15:36:08.859334 11457 raft_consensus.cc:3037] T 00000000000000000000000000000000 P d36c8ca8de404275bc63795e17cb197e [term 1 FOLLOWER]: Advancing to term 2
I20260809 15:36:08.860649 11457 raft_consensus.cc:504] T 00000000000000000000000000000000 P d36c8ca8de404275bc63795e17cb197e [term 2 FOLLOWER]: Starting leader election with config: opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "b7ef4cef863642f19bd9f01ae2792fe3" member_type: VOTER last_known_addr { host: "127.3.89.254" port: 46611 } } peers { permanent_uuid: "d36c8ca8de404275bc63795e17cb197e" member_type: VOTER last_known_addr { host: "127.3.89.253" port: 42075 } attrs { promote: false } } peers { permanent_uuid: "4bb903b269ca47afa0e1d4334950ab94" member_type: VOTER last_known_addr { host: "127.3.89.252" port: 44407 } attrs { promote: false } }
I20260809 15:36:08.860834 11464 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:36:08.856646 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.89.253:42075 (local address 127.0.0.1:60280)
0809 15:36:08.856880 (+ 234us) negotiation.cc:107] Waiting for socket to connect
0809 15:36:08.856891 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0809 15:36:08.857013 (+ 122us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 15:36:08.857403 (+ 390us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 15:36:08.857406 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 15:36:08.857423 (+ 17us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 15:36:08.857530 (+ 107us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:36:08.857540 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:36:08.858526 (+ 986us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:36:08.858529 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:36:08.859639 (+ 1110us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:36:08.859649 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:36:08.860458 (+ 809us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:36:08.860461 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:36:08.860533 (+ 72us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:36:08.860553 (+ 20us) client_negotiation.cc:769] Sending connection context
0809 15:36:08.860627 (+ 74us) client_negotiation.cc:240] Negotiation successful
0809 15:36:08.860676 (+ 49us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":186,"thread_start_us":57,"threads_started":1}
I20260809 15:36:08.860842 11465 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:36:08.856783 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60280 (local address 127.3.89.253:42075)
0809 15:36:08.857083 (+ 300us) server_negotiation.cc:206] Beginning negotiation
0809 15:36:08.857095 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0809 15:36:08.857109 (+ 14us) server_negotiation.cc:407] Connection header received
0809 15:36:08.857144 (+ 35us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 15:36:08.857147 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 15:36:08.857210 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 15:36:08.857289 (+ 79us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 15:36:08.857670 (+ 381us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:36:08.858419 (+ 749us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:36:08.859759 (+ 1340us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:36:08.860320 (+ 561us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:36:08.860450 (+ 130us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:36:08.860536 (+ 86us) server_negotiation.cc:1035] Waiting for connection context
0809 15:36:08.860667 (+ 131us) server_negotiation.cc:299] Negotiation successful
0809 15:36:08.860737 (+ 70us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":225,"thread_start_us":111,"threads_started":1}
I20260809 15:36:08.861029 11457 leader_election.cc:288] T 00000000000000000000000000000000 P d36c8ca8de404275bc63795e17cb197e [CANDIDATE]: Term 2 election: Requested vote from peers b7ef4cef863642f19bd9f01ae2792fe3 (127.3.89.254:46611), 4bb903b269ca47afa0e1d4334950ab94 (127.3.89.252:44407)
I20260809 15:36:08.861310 10686 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "4bb903b269ca47afa0e1d4334950ab94" candidate_term: 2 candidate_status { last_received { term: 1 index: 12 } } ignore_live_leader: false dest_uuid: "d36c8ca8de404275bc63795e17cb197e" is_pre_election: true
I20260809 15:36:08.861462 11459 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:36:08.861228 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.89.254:46611 (local address 127.0.0.1:53426)
0809 15:36:08.861356 (+ 128us) negotiation.cc:107] Waiting for socket to connect
0809 15:36:08.861409 (+ 53us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.89.254:46611: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":58}
I20260809 15:36:08.861512 10686 raft_consensus.cc:2379] T 00000000000000000000000000000000 P d36c8ca8de404275bc63795e17cb197e [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4bb903b269ca47afa0e1d4334950ab94 in current term 2: Already voted for candidate d36c8ca8de404275bc63795e17cb197e in this term.
I20260809 15:36:08.861891 10890 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "d36c8ca8de404275bc63795e17cb197e" candidate_term: 2 candidate_status { last_received { term: 1 index: 12 } } ignore_live_leader: false dest_uuid: "4bb903b269ca47afa0e1d4334950ab94"
I20260809 15:36:08.861901 10864 leader_election.cc:302] T 00000000000000000000000000000000 P 4bb903b269ca47afa0e1d4334950ab94 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4bb903b269ca47afa0e1d4334950ab94; no voters: b7ef4cef863642f19bd9f01ae2792fe3, d36c8ca8de404275bc63795e17cb197e
I20260809 15:36:08.862015 10890 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 4bb903b269ca47afa0e1d4334950ab94 [term 1 FOLLOWER]: Advancing to term 2
I20260809 15:36:08.862154 11459 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:36:08.861920 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.89.254:46611 (local address 127.0.0.1:53440)
0809 15:36:08.862041 (+ 121us) negotiation.cc:107] Waiting for socket to connect
0809 15:36:08.862093 (+ 52us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.89.254:46611: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":55}
W20260809 15:36:08.862304 10661 leader_election.cc:334] T 00000000000000000000000000000000 P d36c8ca8de404275bc63795e17cb197e [CANDIDATE]: Term 2 election: RPC error from VoteRequest() call to peer b7ef4cef863642f19bd9f01ae2792fe3 (127.3.89.254:46611): Network error: Client connection negotiation failed: client connection to 127.3.89.254:46611: connect: Connection refused (error 111)
I20260809 15:36:08.863106 10890 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 4bb903b269ca47afa0e1d4334950ab94 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d36c8ca8de404275bc63795e17cb197e in term 2.
I20260809 15:36:08.863205 11462 raft_consensus.cc:2726] T 00000000000000000000000000000000 P 4bb903b269ca47afa0e1d4334950ab94 [term 2 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260809 15:36:08.863456 10662 leader_election.cc:302] T 00000000000000000000000000000000 P d36c8ca8de404275bc63795e17cb197e [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4bb903b269ca47afa0e1d4334950ab94, d36c8ca8de404275bc63795e17cb197e; no voters: b7ef4cef863642f19bd9f01ae2792fe3
I20260809 15:36:08.863615 11457 raft_consensus.cc:2781] T 00000000000000000000000000000000 P d36c8ca8de404275bc63795e17cb197e [term 2 FOLLOWER]: Leader election won for term 2
I20260809 15:36:08.863821 11457 raft_consensus.cc:686] T 00000000000000000000000000000000 P d36c8ca8de404275bc63795e17cb197e [term 2 LEADER]: Becoming Leader. State: Replica: d36c8ca8de404275bc63795e17cb197e, State: Running, Role: LEADER
I20260809 15:36:08.864064 11457 consensus_queue.cc:237] T 00000000000000000000000000000000 P d36c8ca8de404275bc63795e17cb197e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 1.12, Last appended by leader: 12, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "b7ef4cef863642f19bd9f01ae2792fe3" member_type: VOTER last_known_addr { host: "127.3.89.254" port: 46611 } } peers { permanent_uuid: "d36c8ca8de404275bc63795e17cb197e" member_type: VOTER last_known_addr { host: "127.3.89.253" port: 42075 } attrs { promote: false } } peers { permanent_uuid: "4bb903b269ca47afa0e1d4334950ab94" member_type: VOTER last_known_addr { host: "127.3.89.252" port: 44407 } attrs { promote: false } }
I20260809 15:36:08.865399 11467 sys_catalog.cc:455] T 00000000000000000000000000000000 P d36c8ca8de404275bc63795e17cb197e [sys.catalog]: SysCatalogTable state changed. Reason: New leader d36c8ca8de404275bc63795e17cb197e. Latest consensus state: current_term: 2 leader_uuid: "d36c8ca8de404275bc63795e17cb197e" committed_config { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "b7ef4cef863642f19bd9f01ae2792fe3" member_type: VOTER last_known_addr { host: "127.3.89.254" port: 46611 } } peers { permanent_uuid: "d36c8ca8de404275bc63795e17cb197e" member_type: VOTER last_known_addr { host: "127.3.89.253" port: 42075 } attrs { promote: false } } peers { permanent_uuid: "4bb903b269ca47afa0e1d4334950ab94" member_type: VOTER last_known_addr { host: "127.3.89.252" port: 44407 } attrs { promote: false } } }
I20260809 15:36:08.865521 11467 sys_catalog.cc:458] T 00000000000000000000000000000000 P d36c8ca8de404275bc63795e17cb197e [sys.catalog]: This master's current role is: LEADER
I20260809 15:36:08.865818 11469 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 15:36:08.868194 11469 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 15:36:08.868607 11469 catalog_manager.cc:1259] Loaded cluster ID: 1422be81015249aead82021a784a5898
I20260809 15:36:08.868654 11469 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 15:36:08.869305 11469 catalog_manager.cc:1504] Loading token signing keys...
I20260809 15:36:08.869714 11469 catalog_manager.cc:5879] T 00000000000000000000000000000000 P d36c8ca8de404275bc63795e17cb197e: Loaded TSK: 0
I20260809 15:36:08.870474 11469 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260809 15:36:08.906714 10675 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:36532:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260809 15:36:08.909830 10675 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260809 15:36:08.912984 11459 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:36:08.912703 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.89.254:46611 (local address 127.0.0.1:53450)
0809 15:36:08.912843 (+ 140us) negotiation.cc:107] Waiting for socket to connect
0809 15:36:08.912906 (+ 63us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.89.254:46611: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":50}
I20260809 15:36:08.913043 10890 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 4bb903b269ca47afa0e1d4334950ab94 [term 2 FOLLOWER]: Refusing update from remote peer d36c8ca8de404275bc63795e17cb197e: Log matching property violated. Preceding OpId in replica: term: 1 index: 12. Preceding OpId from leader: term: 2 index: 14. (index mismatch)
I20260809 15:36:08.913522 11457 consensus_queue.cc:1035] T 00000000000000000000000000000000 P d36c8ca8de404275bc63795e17cb197e [LEADER]: Connected to new peer: Peer: permanent_uuid: "4bb903b269ca47afa0e1d4334950ab94" member_type: VOTER last_known_addr { host: "127.3.89.252" port: 44407 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.000s
I20260809 15:36:08.913750 11459 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:36:08.913501 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.89.254:46611 (local address 127.0.0.1:53464)
0809 15:36:08.913609 (+ 108us) negotiation.cc:107] Waiting for socket to connect
0809 15:36:08.913663 (+ 54us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.89.254:46611: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":46}
W20260809 15:36:08.913851 10661 consensus_peers.cc:487] T 00000000000000000000000000000000 P d36c8ca8de404275bc63795e17cb197e -> Peer b7ef4cef863642f19bd9f01ae2792fe3 (127.3.89.254:46611): Couldn't send request to peer b7ef4cef863642f19bd9f01ae2792fe3. Status: Network error: Client connection negotiation failed: client connection to 127.3.89.254:46611: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260809 15:36:08.915045 11462 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4bb903b269ca47afa0e1d4334950ab94 [sys.catalog]: SysCatalogTable state changed. Reason: New leader d36c8ca8de404275bc63795e17cb197e. Latest consensus state: current_term: 2 leader_uuid: "d36c8ca8de404275bc63795e17cb197e" committed_config { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "b7ef4cef863642f19bd9f01ae2792fe3" member_type: VOTER last_known_addr { host: "127.3.89.254" port: 46611 } } peers { permanent_uuid: "d36c8ca8de404275bc63795e17cb197e" member_type: VOTER last_known_addr { host: "127.3.89.253" port: 42075 } attrs { promote: false } } peers { permanent_uuid: "4bb903b269ca47afa0e1d4334950ab94" member_type: VOTER last_known_addr { host: "127.3.89.252" port: 44407 } attrs { promote: false } } }
I20260809 15:36:08.915161 11462 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4bb903b269ca47afa0e1d4334950ab94 [sys.catalog]: This master's current role is: FOLLOWER
I20260809 15:36:08.916436 11467 sys_catalog.cc:455] T 00000000000000000000000000000000 P d36c8ca8de404275bc63795e17cb197e [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 2 leader_uuid: "d36c8ca8de404275bc63795e17cb197e" committed_config { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "b7ef4cef863642f19bd9f01ae2792fe3" member_type: VOTER last_known_addr { host: "127.3.89.254" port: 46611 } } peers { permanent_uuid: "d36c8ca8de404275bc63795e17cb197e" member_type: VOTER last_known_addr { host: "127.3.89.253" port: 42075 } attrs { promote: false } } peers { permanent_uuid: "4bb903b269ca47afa0e1d4334950ab94" member_type: VOTER last_known_addr { host: "127.3.89.252" port: 44407 } attrs { promote: false } } }
I20260809 15:36:08.916779 11467 sys_catalog.cc:458] T 00000000000000000000000000000000 P d36c8ca8de404275bc63795e17cb197e [sys.catalog]: This master's current role is: LEADER
I20260809 15:36:08.916738 11462 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4bb903b269ca47afa0e1d4334950ab94 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 2 leader_uuid: "d36c8ca8de404275bc63795e17cb197e" committed_config { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "b7ef4cef863642f19bd9f01ae2792fe3" member_type: VOTER last_known_addr { host: "127.3.89.254" port: 46611 } } peers { permanent_uuid: "d36c8ca8de404275bc63795e17cb197e" member_type: VOTER last_known_addr { host: "127.3.89.253" port: 42075 } attrs { promote: false } } peers { permanent_uuid: "4bb903b269ca47afa0e1d4334950ab94" member_type: VOTER last_known_addr { host: "127.3.89.252" port: 44407 } attrs { promote: false } } }
I20260809 15:36:08.916872 11462 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4bb903b269ca47afa0e1d4334950ab94 [sys.catalog]: This master's current role is: FOLLOWER
I20260809 15:36:08.927767 11459 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:36:08.922616 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.89.193:44753 (local address 127.0.0.1:39450)
0809 15:36:08.922727 (+ 111us) negotiation.cc:107] Waiting for socket to connect
0809 15:36:08.922753 (+ 26us) client_negotiation.cc:174] Beginning negotiation
0809 15:36:08.922837 (+ 84us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 15:36:08.923624 (+ 787us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 15:36:08.923629 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 15:36:08.923652 (+ 23us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 15:36:08.923758 (+ 106us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:36:08.923767 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:36:08.924945 (+ 1178us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:36:08.924948 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:36:08.926214 (+ 1266us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:36:08.926224 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:36:08.927352 (+ 1128us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:36:08.927356 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:36:08.927508 (+ 152us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:36:08.927535 (+ 27us) client_negotiation.cc:769] Sending connection context
0809 15:36:08.927605 (+ 70us) client_negotiation.cc:240] Negotiation successful
0809 15:36:08.927667 (+ 62us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":43,"mutex_wait_us":135}
I20260809 15:36:08.927778 11475 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:36:08.922772 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39450 (local address 127.3.89.193:44753)
0809 15:36:08.923333 (+ 561us) server_negotiation.cc:206] Beginning negotiation
0809 15:36:08.923341 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0809 15:36:08.923357 (+ 16us) server_negotiation.cc:407] Connection header received
0809 15:36:08.923412 (+ 55us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 15:36:08.923415 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 15:36:08.923481 (+ 66us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 15:36:08.923881 (+ 400us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 15:36:08.924003 (+ 122us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:36:08.924811 (+ 808us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:36:08.926565 (+ 1754us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:36:08.927213 (+ 648us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:36:08.927513 (+ 300us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:36:08.927577 (+ 64us) server_negotiation.cc:1035] Waiting for connection context
0809 15:36:08.927614 (+ 37us) server_negotiation.cc:299] Negotiation successful
0809 15:36:08.927669 (+ 55us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":469,"thread_start_us":106,"threads_started":1}
I20260809 15:36:08.928303 11458 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:36:08.922870 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.89.194:36959 (local address 127.0.0.1:52688)
0809 15:36:08.923062 (+ 192us) negotiation.cc:107] Waiting for socket to connect
0809 15:36:08.923077 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0809 15:36:08.923150 (+ 73us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 15:36:08.923851 (+ 701us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 15:36:08.923854 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 15:36:08.923872 (+ 18us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 15:36:08.923979 (+ 107us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:36:08.923984 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:36:08.924898 (+ 914us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:36:08.924903 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:36:08.926418 (+ 1515us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:36:08.926427 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:36:08.927336 (+ 909us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:36:08.927339 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:36:08.927450 (+ 111us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:36:08.928040 (+ 590us) client_negotiation.cc:769] Sending connection context
0809 15:36:08.928111 (+ 71us) client_negotiation.cc:240] Negotiation successful
0809 15:36:08.928187 (+ 76us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":124,"mutex_wait_us":744}
I20260809 15:36:08.928449 11478 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:36:08.923243 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.89.195:36983 (local address 127.0.0.1:39062)
0809 15:36:08.923492 (+ 249us) negotiation.cc:107] Waiting for socket to connect
0809 15:36:08.923504 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0809 15:36:08.923584 (+ 80us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 15:36:08.924201 (+ 617us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 15:36:08.924204 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 15:36:08.924219 (+ 15us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 15:36:08.924306 (+ 87us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:36:08.924312 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:36:08.925224 (+ 912us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:36:08.925229 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:36:08.926307 (+ 1078us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:36:08.926319 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:36:08.927444 (+ 1125us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:36:08.927447 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:36:08.928112 (+ 665us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:36:08.928137 (+ 25us) client_negotiation.cc:769] Sending connection context
0809 15:36:08.928209 (+ 72us) client_negotiation.cc:240] Negotiation successful
0809 15:36:08.928284 (+ 75us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":196,"mutex_wait_us":522,"thread_start_us":109,"threads_started":1}
I20260809 15:36:08.928468 11004 tablet_service.cc:1467] Processing CreateTablet for tablet 03c0c332b19c447e850d022ebb8199ab (DEFAULT_TABLE table=test-table [id=460765796c65422bb55bdfe492f14a25]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 15:36:08.928759 11004 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 03c0c332b19c447e850d022ebb8199ab. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 15:36:08.928855 11476 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:36:08.923141 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52688 (local address 127.3.89.194:36959)
0809 15:36:08.923368 (+ 227us) server_negotiation.cc:206] Beginning negotiation
0809 15:36:08.923374 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 15:36:08.923390 (+ 16us) server_negotiation.cc:407] Connection header received
0809 15:36:08.923443 (+ 53us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 15:36:08.923446 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 15:36:08.923492 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 15:36:08.924081 (+ 589us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 15:36:08.924208 (+ 127us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:36:08.924789 (+ 581us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:36:08.926546 (+ 1757us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:36:08.927205 (+ 659us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:36:08.927328 (+ 123us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:36:08.927393 (+ 65us) server_negotiation.cc:1035] Waiting for connection context
0809 15:36:08.928633 (+ 1240us) server_negotiation.cc:299] Negotiation successful
0809 15:36:08.928719 (+ 86us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":161,"thread_start_us":95,"threads_started":1}
I20260809 15:36:08.929203 11479 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:36:08.923708 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39062 (local address 127.3.89.195:36983)
0809 15:36:08.923951 (+ 243us) server_negotiation.cc:206] Beginning negotiation
0809 15:36:08.923955 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 15:36:08.923983 (+ 28us) server_negotiation.cc:407] Connection header received
0809 15:36:08.924035 (+ 52us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 15:36:08.924039 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 15:36:08.924086 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 15:36:08.924173 (+ 87us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 15:36:08.924427 (+ 254us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:36:08.925117 (+ 690us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:36:08.926441 (+ 1324us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:36:08.927213 (+ 772us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:36:08.927332 (+ 119us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:36:08.927393 (+ 61us) server_negotiation.cc:1035] Waiting for connection context
0809 15:36:08.928242 (+ 849us) server_negotiation.cc:299] Negotiation successful
0809 15:36:08.928326 (+ 84us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":179,"thread_start_us":73,"threads_started":1}
I20260809 15:36:08.929366 11155 tablet_service.cc:1467] Processing CreateTablet for tablet 03c0c332b19c447e850d022ebb8199ab (DEFAULT_TABLE table=test-table [id=460765796c65422bb55bdfe492f14a25]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 15:36:08.929637 11155 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 03c0c332b19c447e850d022ebb8199ab. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 15:36:08.929726 11306 tablet_service.cc:1467] Processing CreateTablet for tablet 03c0c332b19c447e850d022ebb8199ab (DEFAULT_TABLE table=test-table [id=460765796c65422bb55bdfe492f14a25]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 15:36:08.930024 11306 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 03c0c332b19c447e850d022ebb8199ab. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 15:36:08.931476 11480 tablet_bootstrap.cc:492] T 03c0c332b19c447e850d022ebb8199ab P 5cbab4c3b10e42448c1d2d3cd71ea8be: Bootstrap starting.
I20260809 15:36:08.932230 11481 tablet_bootstrap.cc:492] T 03c0c332b19c447e850d022ebb8199ab P 7f9e63d41b3e46d9a25fb76d9463f92c: Bootstrap starting.
I20260809 15:36:08.932569 11480 tablet_bootstrap.cc:654] T 03c0c332b19c447e850d022ebb8199ab P 5cbab4c3b10e42448c1d2d3cd71ea8be: Neither blocks nor log segments found. Creating new log.
I20260809 15:36:08.932577 11482 tablet_bootstrap.cc:492] T 03c0c332b19c447e850d022ebb8199ab P 4d0727bc39574ad3b523c62e60bb1615: Bootstrap starting.
I20260809 15:36:08.933122 11481 tablet_bootstrap.cc:654] T 03c0c332b19c447e850d022ebb8199ab P 7f9e63d41b3e46d9a25fb76d9463f92c: Neither blocks nor log segments found. Creating new log.
I20260809 15:36:08.933486 11482 tablet_bootstrap.cc:654] T 03c0c332b19c447e850d022ebb8199ab P 4d0727bc39574ad3b523c62e60bb1615: Neither blocks nor log segments found. Creating new log.
I20260809 15:36:08.933928 11480 tablet_bootstrap.cc:492] T 03c0c332b19c447e850d022ebb8199ab P 5cbab4c3b10e42448c1d2d3cd71ea8be: No bootstrap required, opened a new log
I20260809 15:36:08.934019 11480 ts_tablet_manager.cc:1397] T 03c0c332b19c447e850d022ebb8199ab P 5cbab4c3b10e42448c1d2d3cd71ea8be: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260809 15:36:08.934319 11481 tablet_bootstrap.cc:492] T 03c0c332b19c447e850d022ebb8199ab P 7f9e63d41b3e46d9a25fb76d9463f92c: No bootstrap required, opened a new log
I20260809 15:36:08.934414 11481 ts_tablet_manager.cc:1397] T 03c0c332b19c447e850d022ebb8199ab P 7f9e63d41b3e46d9a25fb76d9463f92c: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260809 15:36:08.934507 11480 raft_consensus.cc:348] T 03c0c332b19c447e850d022ebb8199ab P 5cbab4c3b10e42448c1d2d3cd71ea8be [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5cbab4c3b10e42448c1d2d3cd71ea8be" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 44753 } } peers { permanent_uuid: "7f9e63d41b3e46d9a25fb76d9463f92c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 36959 } } peers { permanent_uuid: "4d0727bc39574ad3b523c62e60bb1615" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 36983 } }
I20260809 15:36:08.934579 11482 tablet_bootstrap.cc:492] T 03c0c332b19c447e850d022ebb8199ab P 4d0727bc39574ad3b523c62e60bb1615: No bootstrap required, opened a new log
I20260809 15:36:08.934643 11480 raft_consensus.cc:374] T 03c0c332b19c447e850d022ebb8199ab P 5cbab4c3b10e42448c1d2d3cd71ea8be [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 15:36:08.934674 11482 ts_tablet_manager.cc:1397] T 03c0c332b19c447e850d022ebb8199ab P 4d0727bc39574ad3b523c62e60bb1615: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260809 15:36:08.934684 11480 raft_consensus.cc:729] T 03c0c332b19c447e850d022ebb8199ab P 5cbab4c3b10e42448c1d2d3cd71ea8be [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5cbab4c3b10e42448c1d2d3cd71ea8be, State: Initialized, Role: FOLLOWER
I20260809 15:36:08.934837 11480 consensus_queue.cc:260] T 03c0c332b19c447e850d022ebb8199ab P 5cbab4c3b10e42448c1d2d3cd71ea8be [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5cbab4c3b10e42448c1d2d3cd71ea8be" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 44753 } } peers { permanent_uuid: "7f9e63d41b3e46d9a25fb76d9463f92c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 36959 } } peers { permanent_uuid: "4d0727bc39574ad3b523c62e60bb1615" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 36983 } }
I20260809 15:36:08.934831 11481 raft_consensus.cc:348] T 03c0c332b19c447e850d022ebb8199ab P 7f9e63d41b3e46d9a25fb76d9463f92c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5cbab4c3b10e42448c1d2d3cd71ea8be" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 44753 } } peers { permanent_uuid: "7f9e63d41b3e46d9a25fb76d9463f92c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 36959 } } peers { permanent_uuid: "4d0727bc39574ad3b523c62e60bb1615" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 36983 } }
I20260809 15:36:08.934948 11481 raft_consensus.cc:374] T 03c0c332b19c447e850d022ebb8199ab P 7f9e63d41b3e46d9a25fb76d9463f92c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 15:36:08.935019 11481 raft_consensus.cc:729] T 03c0c332b19c447e850d022ebb8199ab P 7f9e63d41b3e46d9a25fb76d9463f92c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7f9e63d41b3e46d9a25fb76d9463f92c, State: Initialized, Role: FOLLOWER
I20260809 15:36:08.935148 11481 consensus_queue.cc:260] T 03c0c332b19c447e850d022ebb8199ab P 7f9e63d41b3e46d9a25fb76d9463f92c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5cbab4c3b10e42448c1d2d3cd71ea8be" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 44753 } } peers { permanent_uuid: "7f9e63d41b3e46d9a25fb76d9463f92c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 36959 } } peers { permanent_uuid: "4d0727bc39574ad3b523c62e60bb1615" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 36983 } }
I20260809 15:36:08.935144 11482 raft_consensus.cc:348] T 03c0c332b19c447e850d022ebb8199ab P 4d0727bc39574ad3b523c62e60bb1615 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5cbab4c3b10e42448c1d2d3cd71ea8be" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 44753 } } peers { permanent_uuid: "7f9e63d41b3e46d9a25fb76d9463f92c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 36959 } } peers { permanent_uuid: "4d0727bc39574ad3b523c62e60bb1615" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 36983 } }
I20260809 15:36:08.935215 11480 ts_tablet_manager.cc:1428] T 03c0c332b19c447e850d022ebb8199ab P 5cbab4c3b10e42448c1d2d3cd71ea8be: Time spent starting tablet: real 0.001s user 0.002s sys 0.000s
I20260809 15:36:08.935245 11482 raft_consensus.cc:374] T 03c0c332b19c447e850d022ebb8199ab P 4d0727bc39574ad3b523c62e60bb1615 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 15:36:08.935277 11482 raft_consensus.cc:729] T 03c0c332b19c447e850d022ebb8199ab P 4d0727bc39574ad3b523c62e60bb1615 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4d0727bc39574ad3b523c62e60bb1615, State: Initialized, Role: FOLLOWER
I20260809 15:36:08.935410 11482 consensus_queue.cc:260] T 03c0c332b19c447e850d022ebb8199ab P 4d0727bc39574ad3b523c62e60bb1615 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5cbab4c3b10e42448c1d2d3cd71ea8be" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 44753 } } peers { permanent_uuid: "7f9e63d41b3e46d9a25fb76d9463f92c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 36959 } } peers { permanent_uuid: "4d0727bc39574ad3b523c62e60bb1615" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 36983 } }
I20260809 15:36:08.935503 11481 ts_tablet_manager.cc:1428] T 03c0c332b19c447e850d022ebb8199ab P 7f9e63d41b3e46d9a25fb76d9463f92c: Time spent starting tablet: real 0.001s user 0.000s sys 0.001s
I20260809 15:36:08.935716 11482 ts_tablet_manager.cc:1428] T 03c0c332b19c447e850d022ebb8199ab P 4d0727bc39574ad3b523c62e60bb1615: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260809 15:36:08.937259 11491 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:36:08.936874 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.89.254:46611 (local address 127.3.89.195:55729)
0809 15:36:08.937114 (+ 240us) negotiation.cc:107] Waiting for socket to connect
0809 15:36:08.937174 (+ 60us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.89.254:46611: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":168,"thread_start_us":96,"threads_started":1}
I20260809 15:36:08.937832 11492 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:36:08.937500 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.89.254:46611 (local address 127.3.89.193:36469)
0809 15:36:08.937717 (+ 217us) negotiation.cc:107] Waiting for socket to connect
0809 15:36:08.937763 (+ 46us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.89.254:46611: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":161,"thread_start_us":96,"threads_started":1}
I20260809 15:36:08.938062 11490 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:36:08.936585 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.89.254:46611 (local address 127.3.89.194:56735)
0809 15:36:08.937918 (+ 1333us) negotiation.cc:107] Waiting for socket to connect
0809 15:36:08.937975 (+ 57us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.89.254:46611: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":1261,"thread_start_us":93,"threads_started":1}
I20260809 15:36:08.939924 11232 heartbeater.cc:503] Master 127.3.89.253:42075 was elected leader, sending a full tablet report...
I20260809 15:36:08.939970 11490 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:36:08.939713 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.89.254:46611 (local address 127.3.89.194:58861)
0809 15:36:08.939841 (+ 128us) negotiation.cc:107] Waiting for socket to connect
0809 15:36:08.939895 (+ 54us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.89.254:46611: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":47}
I20260809 15:36:08.940107 11380 heartbeater.cc:503] Master 127.3.89.253:42075 was elected leader, sending a full tablet report...
I20260809 15:36:08.940296 11491 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:36:08.939956 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.89.254:46611 (local address 127.3.89.195:37139)
0809 15:36:08.940153 (+ 197us) negotiation.cc:107] Waiting for socket to connect
0809 15:36:08.940229 (+ 76us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.89.254:46611: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":125}
I20260809 15:36:08.941502 11080 heartbeater.cc:503] Master 127.3.89.253:42075 was elected leader, sending a full tablet report...
I20260809 15:36:08.941608 11492 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:36:08.941345 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.89.254:46611 (local address 127.3.89.193:34231)
0809 15:36:08.941451 (+ 106us) negotiation.cc:107] Waiting for socket to connect
0809 15:36:08.941505 (+ 54us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.89.254:46611: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":30}
I20260809 15:36:09.273754 11488 raft_consensus.cc:482] T 03c0c332b19c447e850d022ebb8199ab P 7f9e63d41b3e46d9a25fb76d9463f92c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 15:36:09.273942 11488 raft_consensus.cc:504] T 03c0c332b19c447e850d022ebb8199ab P 7f9e63d41b3e46d9a25fb76d9463f92c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5cbab4c3b10e42448c1d2d3cd71ea8be" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 44753 } } peers { permanent_uuid: "7f9e63d41b3e46d9a25fb76d9463f92c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 36959 } } peers { permanent_uuid: "4d0727bc39574ad3b523c62e60bb1615" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 36983 } }
I20260809 15:36:09.274477 11488 leader_election.cc:288] T 03c0c332b19c447e850d022ebb8199ab P 7f9e63d41b3e46d9a25fb76d9463f92c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 5cbab4c3b10e42448c1d2d3cd71ea8be (127.3.89.193:44753), 4d0727bc39574ad3b523c62e60bb1615 (127.3.89.195:36983)
I20260809 15:36:09.275027 11329 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "03c0c332b19c447e850d022ebb8199ab" candidate_uuid: "7f9e63d41b3e46d9a25fb76d9463f92c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4d0727bc39574ad3b523c62e60bb1615" is_pre_election: true
I20260809 15:36:09.275046 11029 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "03c0c332b19c447e850d022ebb8199ab" candidate_uuid: "7f9e63d41b3e46d9a25fb76d9463f92c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5cbab4c3b10e42448c1d2d3cd71ea8be" is_pre_election: true
I20260809 15:36:09.275199 11029 raft_consensus.cc:2454] T 03c0c332b19c447e850d022ebb8199ab P 5cbab4c3b10e42448c1d2d3cd71ea8be [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7f9e63d41b3e46d9a25fb76d9463f92c in term 0.
I20260809 15:36:09.275211 11329 raft_consensus.cc:2454] T 03c0c332b19c447e850d022ebb8199ab P 4d0727bc39574ad3b523c62e60bb1615 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7f9e63d41b3e46d9a25fb76d9463f92c in term 0.
I20260809 15:36:09.275550 11103 leader_election.cc:302] T 03c0c332b19c447e850d022ebb8199ab P 7f9e63d41b3e46d9a25fb76d9463f92c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5cbab4c3b10e42448c1d2d3cd71ea8be, 7f9e63d41b3e46d9a25fb76d9463f92c; no voters:
I20260809 15:36:09.275811 11488 raft_consensus.cc:2781] T 03c0c332b19c447e850d022ebb8199ab P 7f9e63d41b3e46d9a25fb76d9463f92c [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260809 15:36:09.275862 11488 raft_consensus.cc:482] T 03c0c332b19c447e850d022ebb8199ab P 7f9e63d41b3e46d9a25fb76d9463f92c [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 15:36:09.275918 11488 raft_consensus.cc:3037] T 03c0c332b19c447e850d022ebb8199ab P 7f9e63d41b3e46d9a25fb76d9463f92c [term 0 FOLLOWER]: Advancing to term 1
I20260809 15:36:09.277014 11488 raft_consensus.cc:504] T 03c0c332b19c447e850d022ebb8199ab P 7f9e63d41b3e46d9a25fb76d9463f92c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5cbab4c3b10e42448c1d2d3cd71ea8be" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 44753 } } peers { permanent_uuid: "7f9e63d41b3e46d9a25fb76d9463f92c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 36959 } } peers { permanent_uuid: "4d0727bc39574ad3b523c62e60bb1615" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 36983 } }
I20260809 15:36:09.277405 11488 leader_election.cc:288] T 03c0c332b19c447e850d022ebb8199ab P 7f9e63d41b3e46d9a25fb76d9463f92c [CANDIDATE]: Term 1 election: Requested vote from peers 5cbab4c3b10e42448c1d2d3cd71ea8be (127.3.89.193:44753), 4d0727bc39574ad3b523c62e60bb1615 (127.3.89.195:36983)
I20260809 15:36:09.277738 11329 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "03c0c332b19c447e850d022ebb8199ab" candidate_uuid: "7f9e63d41b3e46d9a25fb76d9463f92c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4d0727bc39574ad3b523c62e60bb1615"
I20260809 15:36:09.277766 11029 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "03c0c332b19c447e850d022ebb8199ab" candidate_uuid: "7f9e63d41b3e46d9a25fb76d9463f92c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5cbab4c3b10e42448c1d2d3cd71ea8be"
I20260809 15:36:09.277868 11329 raft_consensus.cc:3037] T 03c0c332b19c447e850d022ebb8199ab P 4d0727bc39574ad3b523c62e60bb1615 [term 0 FOLLOWER]: Advancing to term 1
I20260809 15:36:09.277907 11029 raft_consensus.cc:3037] T 03c0c332b19c447e850d022ebb8199ab P 5cbab4c3b10e42448c1d2d3cd71ea8be [term 0 FOLLOWER]: Advancing to term 1
I20260809 15:36:09.278843 11029 raft_consensus.cc:2454] T 03c0c332b19c447e850d022ebb8199ab P 5cbab4c3b10e42448c1d2d3cd71ea8be [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7f9e63d41b3e46d9a25fb76d9463f92c in term 1.
I20260809 15:36:09.278874 11329 raft_consensus.cc:2454] T 03c0c332b19c447e850d022ebb8199ab P 4d0727bc39574ad3b523c62e60bb1615 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7f9e63d41b3e46d9a25fb76d9463f92c in term 1.
I20260809 15:36:09.279178 11107 leader_election.cc:302] T 03c0c332b19c447e850d022ebb8199ab P 7f9e63d41b3e46d9a25fb76d9463f92c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4d0727bc39574ad3b523c62e60bb1615, 7f9e63d41b3e46d9a25fb76d9463f92c; no voters:
I20260809 15:36:09.279348 11488 raft_consensus.cc:2781] T 03c0c332b19c447e850d022ebb8199ab P 7f9e63d41b3e46d9a25fb76d9463f92c [term 1 FOLLOWER]: Leader election won for term 1
I20260809 15:36:09.279599 11488 raft_consensus.cc:686] T 03c0c332b19c447e850d022ebb8199ab P 7f9e63d41b3e46d9a25fb76d9463f92c [term 1 LEADER]: Becoming Leader. State: Replica: 7f9e63d41b3e46d9a25fb76d9463f92c, State: Running, Role: LEADER
I20260809 15:36:09.279757 11488 consensus_queue.cc:237] T 03c0c332b19c447e850d022ebb8199ab P 7f9e63d41b3e46d9a25fb76d9463f92c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5cbab4c3b10e42448c1d2d3cd71ea8be" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 44753 } } peers { permanent_uuid: "7f9e63d41b3e46d9a25fb76d9463f92c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 36959 } } peers { permanent_uuid: "4d0727bc39574ad3b523c62e60bb1615" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 36983 } }
I20260809 15:36:09.280398 11490 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:36:09.280095 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.89.254:46611 (local address 127.3.89.194:60295)
0809 15:36:09.280230 (+ 135us) negotiation.cc:107] Waiting for socket to connect
0809 15:36:09.280316 (+ 86us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.89.254:46611: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":51}
I20260809 15:36:09.282157 10673 catalog_manager.cc:5495] T 03c0c332b19c447e850d022ebb8199ab P 7f9e63d41b3e46d9a25fb76d9463f92c reported cstate change: term changed from 0 to 1, leader changed from <none> to 7f9e63d41b3e46d9a25fb76d9463f92c (127.3.89.194). New cstate: current_term: 1 leader_uuid: "7f9e63d41b3e46d9a25fb76d9463f92c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5cbab4c3b10e42448c1d2d3cd71ea8be" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 44753 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7f9e63d41b3e46d9a25fb76d9463f92c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 36959 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "4d0727bc39574ad3b523c62e60bb1615" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 36983 } health_report { overall_health: UNKNOWN } } }
I20260809 15:36:09.378041 11476 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:36:09.374856 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52696 (local address 127.3.89.194:36959)
0809 15:36:09.375054 (+ 198us) server_negotiation.cc:206] Beginning negotiation
0809 15:36:09.375060 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 15:36:09.375077 (+ 17us) server_negotiation.cc:407] Connection header received
0809 15:36:09.375128 (+ 51us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 15:36:09.375130 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 15:36:09.375175 (+ 45us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 15:36:09.375259 (+ 84us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 15:36:09.375523 (+ 264us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:36:09.376037 (+ 514us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:36:09.376921 (+ 884us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:36:09.377147 (+ 226us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:36:09.377257 (+ 110us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:36:09.377578 (+ 321us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 15:36:09.377650 (+ 72us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 15:36:09.377730 (+ 80us) server_negotiation.cc:1035] Waiting for connection context
0809 15:36:09.377824 (+ 94us) server_negotiation.cc:299] Negotiation successful
0809 15:36:09.377877 (+ 53us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":62}
I20260809 15:36:09.380244 11029 raft_consensus.cc:1261] T 03c0c332b19c447e850d022ebb8199ab P 5cbab4c3b10e42448c1d2d3cd71ea8be [term 1 FOLLOWER]: Refusing update from remote peer 7f9e63d41b3e46d9a25fb76d9463f92c: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 15:36:09.380273 11329 raft_consensus.cc:1261] T 03c0c332b19c447e850d022ebb8199ab P 4d0727bc39574ad3b523c62e60bb1615 [term 1 FOLLOWER]: Refusing update from remote peer 7f9e63d41b3e46d9a25fb76d9463f92c: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 15:36:09.380734 11488 consensus_queue.cc:1035] T 03c0c332b19c447e850d022ebb8199ab P 7f9e63d41b3e46d9a25fb76d9463f92c [LEADER]: Connected to new peer: Peer: permanent_uuid: "4d0727bc39574ad3b523c62e60bb1615" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 36983 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 15:36:09.380897 11495 consensus_queue.cc:1035] T 03c0c332b19c447e850d022ebb8199ab P 7f9e63d41b3e46d9a25fb76d9463f92c [LEADER]: Connected to new peer: Peer: permanent_uuid: "5cbab4c3b10e42448c1d2d3cd71ea8be" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 44753 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 15:36:09.383122 11491 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:36:09.382783 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.89.254:46611 (local address 127.3.89.195:60329)
0809 15:36:09.382935 (+ 152us) negotiation.cc:107] Waiting for socket to connect
0809 15:36:09.383042 (+ 107us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.89.254:46611: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":82}
I20260809 15:36:09.383853 11492 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:36:09.383586 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.89.254:46611 (local address 127.3.89.193:40035)
0809 15:36:09.383716 (+ 130us) negotiation.cc:107] Waiting for socket to connect
0809 15:36:09.383778 (+ 62us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.89.254:46611: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":54}
I20260809 15:36:09.384578 11499 mvcc.cc:205] Tried to move back new op lower bound from 7316642895377211392 to 7316642894971645952. Current Snapshot: MvccSnapshot[applied={T|T < 7316642895377211392}]
I20260809 15:36:09.386078 11490 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:36:09.385340 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.89.254:46611 (local address 127.3.89.194:47331)
0809 15:36:09.385953 (+ 613us) negotiation.cc:107] Waiting for socket to connect
0809 15:36:09.386008 (+ 55us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.89.254:46611: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":542}
I20260809 15:36:09.387056 11491 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:36:09.386754 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.89.254:46611 (local address 127.3.89.195:49531)
0809 15:36:09.386887 (+ 133us) negotiation.cc:107] Waiting for socket to connect
0809 15:36:09.386935 (+ 48us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.89.254:46611: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":76}
I20260809 15:36:09.387313 11490 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:36:09.386530 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.89.254:46611 (local address 127.3.89.194:50599)
0809 15:36:09.387185 (+ 655us) negotiation.cc:107] Waiting for socket to connect
0809 15:36:09.387234 (+ 49us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.89.254:46611: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":582}
I20260809 15:36:09.387352 11492 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:36:09.387067 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.89.254:46611 (local address 127.3.89.193:59319)
0809 15:36:09.387240 (+ 173us) negotiation.cc:107] Waiting for socket to connect
0809 15:36:09.387293 (+ 53us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.89.254:46611: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":103}
I20260809 15:36:09.392942 10673 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:36532:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 15:36:09.393071 10673 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:36532:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 15:36:09.396598 10673 catalog_manager.cc:5782] T 00000000000000000000000000000000 P d36c8ca8de404275bc63795e17cb197e: Sending DeleteTablet for 3 replicas of tablet 03c0c332b19c447e850d022ebb8199ab
I20260809 15:36:09.397378 11306 tablet_service.cc:1514] Processing DeleteTablet for tablet 03c0c332b19c447e850d022ebb8199ab with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 15:36:09 UTC) from {username='oryx', principal='oryx/127.3.89.253@KRBTEST.COM'} at 127.0.0.1:39062
I20260809 15:36:09.397418 11004 tablet_service.cc:1514] Processing DeleteTablet for tablet 03c0c332b19c447e850d022ebb8199ab with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 15:36:09 UTC) from {username='oryx', principal='oryx/127.3.89.253@KRBTEST.COM'} at 127.0.0.1:39450
I20260809 15:36:09.397404 11155 tablet_service.cc:1514] Processing DeleteTablet for tablet 03c0c332b19c447e850d022ebb8199ab with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 15:36:09 UTC) from {username='oryx', principal='oryx/127.3.89.253@KRBTEST.COM'} at 127.0.0.1:52688
I20260809 15:36:09.397766 11510 tablet_replica.cc:330] T 03c0c332b19c447e850d022ebb8199ab P 5cbab4c3b10e42448c1d2d3cd71ea8be: stopping tablet replica
I20260809 15:36:09.397775 11512 tablet_replica.cc:330] T 03c0c332b19c447e850d022ebb8199ab P 7f9e63d41b3e46d9a25fb76d9463f92c: stopping tablet replica
I20260809 15:36:09.397898 11510 raft_consensus.cc:2229] T 03c0c332b19c447e850d022ebb8199ab P 5cbab4c3b10e42448c1d2d3cd71ea8be [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 15:36:09.397902 11512 raft_consensus.cc:2229] T 03c0c332b19c447e850d022ebb8199ab P 7f9e63d41b3e46d9a25fb76d9463f92c [term 1 LEADER]: Raft consensus shutting down.
I20260809 15:36:09.398051 11510 raft_consensus.cc:2258] T 03c0c332b19c447e850d022ebb8199ab P 5cbab4c3b10e42448c1d2d3cd71ea8be [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 15:36:09.398094 3431 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskm314YR/build/debug/bin/kudu with pid 10834
I20260809 15:36:09.398097 11512 raft_consensus.cc:2258] T 03c0c332b19c447e850d022ebb8199ab P 7f9e63d41b3e46d9a25fb76d9463f92c [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 15:36:09.399151 11511 tablet_replica.cc:330] T 03c0c332b19c447e850d022ebb8199ab P 4d0727bc39574ad3b523c62e60bb1615: stopping tablet replica
I20260809 15:36:09.399287 11511 raft_consensus.cc:2229] T 03c0c332b19c447e850d022ebb8199ab P 4d0727bc39574ad3b523c62e60bb1615 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 15:36:09.399448 11511 raft_consensus.cc:2258] T 03c0c332b19c447e850d022ebb8199ab P 4d0727bc39574ad3b523c62e60bb1615 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 15:36:09.400696 11511 ts_tablet_manager.cc:1905] T 03c0c332b19c447e850d022ebb8199ab P 4d0727bc39574ad3b523c62e60bb1615: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 15:36:09.401392 11512 ts_tablet_manager.cc:1905] T 03c0c332b19c447e850d022ebb8199ab P 7f9e63d41b3e46d9a25fb76d9463f92c: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 15:36:09.403013 11511 ts_tablet_manager.cc:1918] T 03c0c332b19c447e850d022ebb8199ab P 4d0727bc39574ad3b523c62e60bb1615: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 15:36:09.403102 11511 log.cc:1196] T 03c0c332b19c447e850d022ebb8199ab P 4d0727bc39574ad3b523c62e60bb1615: Deleting WAL directory at /tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786289732823074-3431-0/minicluster-data/ts-2/wal/wals/03c0c332b19c447e850d022ebb8199ab
I20260809 15:36:09.403395 11511 ts_tablet_manager.cc:1939] T 03c0c332b19c447e850d022ebb8199ab P 4d0727bc39574ad3b523c62e60bb1615: Deleting consensus metadata
I20260809 15:36:09.404552 11478 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:36:09.404297 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.89.254:46611 (local address 127.0.0.1:53468)
0809 15:36:09.404420 (+ 123us) negotiation.cc:107] Waiting for socket to connect
0809 15:36:09.404483 (+ 63us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.89.254:46611: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":46}
I20260809 15:36:09.404641 11512 ts_tablet_manager.cc:1918] T 03c0c332b19c447e850d022ebb8199ab P 7f9e63d41b3e46d9a25fb76d9463f92c: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 15:36:09.404724 11512 log.cc:1196] T 03c0c332b19c447e850d022ebb8199ab P 7f9e63d41b3e46d9a25fb76d9463f92c: Deleting WAL directory at /tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786289732823074-3431-0/minicluster-data/ts-1/wal/wals/03c0c332b19c447e850d022ebb8199ab
I20260809 15:36:09.405036 11512 ts_tablet_manager.cc:1939] T 03c0c332b19c447e850d022ebb8199ab P 7f9e63d41b3e46d9a25fb76d9463f92c: Deleting consensus metadata
I20260809 15:36:09.405277 11478 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:36:09.405059 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.89.254:46611 (local address 127.0.0.1:53470)
0809 15:36:09.405168 (+ 109us) negotiation.cc:107] Waiting for socket to connect
0809 15:36:09.405221 (+ 53us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.89.254:46611: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":44}
I20260809 15:36:09.405803 10662 catalog_manager.cc:4841] TS 7f9e63d41b3e46d9a25fb76d9463f92c (127.3.89.194:36959): tablet 03c0c332b19c447e850d022ebb8199ab (table test-table [id=460765796c65422bb55bdfe492f14a25]) successfully deleted
I20260809 15:36:09.407272 10662 catalog_manager.cc:4841] TS 4d0727bc39574ad3b523c62e60bb1615 (127.3.89.195:36983): tablet 03c0c332b19c447e850d022ebb8199ab (table test-table [id=460765796c65422bb55bdfe492f14a25]) successfully deleted
I20260809 15:36:09.407670 3431 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskm314YR/build/debug/bin/kudu with pid 11086
I20260809 15:36:09.409332 11478 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:36:09.407890 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.89.193:44753 (local address 127.0.0.1:39462)
0809 15:36:09.409171 (+ 1281us) negotiation.cc:107] Waiting for socket to connect
0809 15:36:09.409247 (+ 76us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.89.193:44753: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":1189}
W20260809 15:36:09.409578 10659 catalog_manager.cc:4584] TS 5cbab4c3b10e42448c1d2d3cd71ea8be (127.3.89.193:44753): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 03c0c332b19c447e850d022ebb8199ab: Network error: Client connection negotiation failed: client connection to 127.3.89.193:44753: connect: Connection refused (error 111)
I20260809 15:36:09.416689 3431 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskm314YR/build/debug/bin/kudu with pid 11237
I20260809 15:36:09.424708 3431 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskm314YR/build/debug/bin/kudu with pid 10522
I20260809 15:36:09.432015 3431 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskm314YR/build/debug/bin/kudu with pid 10637
2026-08-09T15:36:09Z chronyd exiting
[ OK ] SecurityITest.TestNonDefaultPrincipalMultipleMaster (5950 ms)
[ RUN ] SecurityITest.TestNonDefaultPrincipal
Loading random data
Initializing database '/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786289732823074-3431-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 15:36:09 dist-test-slave-2w3w krb5kdc[11515](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 15:36:09 dist-test-slave-2w3w krb5kdc[11515](info): set up 2 sockets
Aug 09 15:36:09 dist-test-slave-2w3w krb5kdc[11515](info): commencing operation
krb5kdc: starting...
W20260809 15:36:10.533006 3431 mini_kdc.cc:121] Time spent starting KDC: real 1.076s user 0.000s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786289732823074-3431-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786289732823074-3431-0/krb5kdc/test-user.keytab.
Aug 09 15:36:10 dist-test-slave-2w3w krb5kdc[11515](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786289770, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T15:36:10Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T15:36:10Z Disabled control of system clock
WARNING: no policy specified for oryx/127.3.89.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.3.89.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.3.89.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786289732823074-3431-0/krb5kdc/oryx_127.3.89.254.keytab.
Entry for principal oryx/127.3.89.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786289732823074-3431-0/krb5kdc/oryx_127.3.89.254.keytab.
I20260809 15:36:10.658088 3431 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskm314YR/build/debug/bin/kudu
/tmp/dist-test-taskm314YR/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786289732823074-3431-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786289732823074-3431-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786289732823074-3431-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786289732823074-3431-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.3.89.254:38093
--webserver_interface=127.3.89.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786289732823074-3431-0/krb5kdc/oryx_127.3.89.254.keytab
--principal=oryx/127.3.89.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.3.89.212:36197
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.3.89.254:38093
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--txn_manager_enabled=true with env {KRB5CCNAME=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786289732823074-3431-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786289732823074-3431-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786289732823074-3431-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 15:36:10.769651 11528 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 15:36:10.769906 11528 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 15:36:10.769948 11528 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 15:36:10.774309 11528 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 15:36:10.774366 11528 flags.cc:425] Enabled experimental flag: --txn_manager_enabled=true
W20260809 15:36:10.774384 11528 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 15:36:10.774401 11528 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 15:36:10.774417 11528 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 15:36:10.774438 11528 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 15:36:10.779729 11528 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.3.89.212:36197
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786289732823074-3431-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786289732823074-3431-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.3.89.254:38093
--txn_manager_enabled=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786289732823074-3431-0/krb5kdc/oryx_127.3.89.254.keytab
--principal=oryx/127.3.89.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.3.89.254:38093
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786289732823074-3431-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.3.89.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.11528
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786289732823074-3431-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 09 Aug 2026 14:46:55 UTC on ccycloud.quasar-yswkmf.root.comops.site
build id 4576540
I20260809 15:36:10.780769 11528 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 15:36:10.782016 11528 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 15:36:10.790602 11534 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 15:36:10.790633 11536 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 15:36:10.790602 11533 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 15:36:10.790931 11528 server_base.cc:1034] running on GCE node
I20260809 15:36:10.791589 11528 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 15:36:10.792640 11528 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 15:36:10.793826 11528 hybrid_clock.cc:648] HybridClock initialized: now 1786289770793809 us; error 33 us; skew 500 ppm
Aug 09 15:36:10 dist-test-slave-2w3w krb5kdc[11515](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786289770, etypes {rep=17 tkt=17 ses=17}, oryx/127.3.89.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 15:36:10.799733 11528 init.cc:377] Logged in from keytab as oryx/127.3.89.254@KRBTEST.COM (short username oryx)
I20260809 15:36:10.800595 11528 webserver.cc:460] Webserver started at http://127.3.89.254:42737/ using document root <none> and password file <none>
I20260809 15:36:10.801157 11528 fs_manager.cc:359] Metadata directory not provided
I20260809 15:36:10.801250 11528 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 15:36:10.801486 11528 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 15:36:10.803547 11528 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786289732823074-3431-0/minicluster-data/master-0/data/instance:
uuid: "6a9afd191e4f40c2a679860b4257c4a7"
format_stamp: "Formatted at 2026-08-09 15:36:10 on dist-test-slave-2w3w"
server_key: "88782218851089d46a416c86279bac97"
server_key_iv: "f83154225acf3e1a68305bba064fc2ca"
server_key_version: "encryptionkey@0"
I20260809 15:36:10.804243 11528 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786289732823074-3431-0/minicluster-data/master-0/wal/instance:
uuid: "6a9afd191e4f40c2a679860b4257c4a7"
format_stamp: "Formatted at 2026-08-09 15:36:10 on dist-test-slave-2w3w"
server_key: "88782218851089d46a416c86279bac97"
server_key_iv: "f83154225acf3e1a68305bba064fc2ca"
server_key_version: "encryptionkey@0"
I20260809 15:36:10.807783 11528 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.004s sys 0.000s
I20260809 15:36:10.810151 11543 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 15:36:10.811324 11528 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260809 15:36:10.811462 11528 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786289732823074-3431-0/minicluster-data/master-0/data,/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786289732823074-3431-0/minicluster-data/master-0/wal
uuid: "6a9afd191e4f40c2a679860b4257c4a7"
format_stamp: "Formatted at 2026-08-09 15:36:10 on dist-test-slave-2w3w"
server_key: "88782218851089d46a416c86279bac97"
server_key_iv: "f83154225acf3e1a68305bba064fc2ca"
server_key_version: "encryptionkey@0"
I20260809 15:36:10.811568 11528 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786289732823074-3431-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786289732823074-3431-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786289732823074-3431-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 15:36:10.823756 11528 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 15:36:10.826386 11528 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 15:36:10.826586 11528 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 15:36:10.834239 11528 rpc_server.cc:307] RPC server started. Bound to: 127.3.89.254:38093
I20260809 15:36:10.834270 11609 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.3.89.254:38093 every 8 connection(s)
I20260809 15:36:10.835151 11528 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786289732823074-3431-0/minicluster-data/master-0/data/info.pb
I20260809 15:36:10.837643 11610 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 15:36:10.842811 11610 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6a9afd191e4f40c2a679860b4257c4a7: Bootstrap starting.
I20260809 15:36:10.844491 3431 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskm314YR/build/debug/bin/kudu as pid 11528
I20260809 15:36:10.844591 3431 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786289732823074-3431-0/minicluster-data/master-0/wal/instance
I20260809 15:36:10.844880 3431 external_mini_cluster.cc:1428] Setting key a2520832af3aa3fe406b46ac0db186bd
I20260809 15:36:10.845342 11610 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 6a9afd191e4f40c2a679860b4257c4a7: Neither blocks nor log segments found. Creating new log.
I20260809 15:36:10.846194 11610 log.cc:824] T 00000000000000000000000000000000 P 6a9afd191e4f40c2a679860b4257c4a7: Log is configured to *not* fsync() on all Append() calls
I20260809 15:36:10.848506 11610 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6a9afd191e4f40c2a679860b4257c4a7: No bootstrap required, opened a new log
I20260809 15:36:10.852687 11610 raft_consensus.cc:348] T 00000000000000000000000000000000 P 6a9afd191e4f40c2a679860b4257c4a7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6a9afd191e4f40c2a679860b4257c4a7" member_type: VOTER last_known_addr { host: "127.3.89.254" port: 38093 } }
I20260809 15:36:10.852939 11610 raft_consensus.cc:374] T 00000000000000000000000000000000 P 6a9afd191e4f40c2a679860b4257c4a7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 15:36:10.853021 11610 raft_consensus.cc:729] T 00000000000000000000000000000000 P 6a9afd191e4f40c2a679860b4257c4a7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6a9afd191e4f40c2a679860b4257c4a7, State: Initialized, Role: FOLLOWER
I20260809 15:36:10.853467 11610 consensus_queue.cc:260] T 00000000000000000000000000000000 P 6a9afd191e4f40c2a679860b4257c4a7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6a9afd191e4f40c2a679860b4257c4a7" member_type: VOTER last_known_addr { host: "127.3.89.254" port: 38093 } }
I20260809 15:36:10.853621 11610 raft_consensus.cc:388] T 00000000000000000000000000000000 P 6a9afd191e4f40c2a679860b4257c4a7 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 15:36:10.853714 11610 raft_consensus.cc:482] T 00000000000000000000000000000000 P 6a9afd191e4f40c2a679860b4257c4a7 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 15:36:10.853834 11610 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 6a9afd191e4f40c2a679860b4257c4a7 [term 0 FOLLOWER]: Advancing to term 1
Aug 09 15:36:10 dist-test-slave-2w3w krb5kdc[11515](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786289770, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.3.89.254@KRBTEST.COM
I20260809 15:36:10.854951 11610 raft_consensus.cc:504] T 00000000000000000000000000000000 P 6a9afd191e4f40c2a679860b4257c4a7 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6a9afd191e4f40c2a679860b4257c4a7" member_type: VOTER last_known_addr { host: "127.3.89.254" port: 38093 } }
I20260809 15:36:10.855289 11610 leader_election.cc:302] T 00000000000000000000000000000000 P 6a9afd191e4f40c2a679860b4257c4a7 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 6a9afd191e4f40c2a679860b4257c4a7; no voters:
I20260809 15:36:10.855618 11610 leader_election.cc:288] T 00000000000000000000000000000000 P 6a9afd191e4f40c2a679860b4257c4a7 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 15:36:10.855691 11615 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 6a9afd191e4f40c2a679860b4257c4a7 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 15:36:10.855952 11615 raft_consensus.cc:686] T 00000000000000000000000000000000 P 6a9afd191e4f40c2a679860b4257c4a7 [term 1 LEADER]: Becoming Leader. State: Replica: 6a9afd191e4f40c2a679860b4257c4a7, State: Running, Role: LEADER
I20260809 15:36:10.856237 11615 consensus_queue.cc:237] T 00000000000000000000000000000000 P 6a9afd191e4f40c2a679860b4257c4a7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6a9afd191e4f40c2a679860b4257c4a7" member_type: VOTER last_known_addr { host: "127.3.89.254" port: 38093 } }
I20260809 15:36:10.856858 11610 sys_catalog.cc:564] T 00000000000000000000000000000000 P 6a9afd191e4f40c2a679860b4257c4a7 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 15:36:10.857990 11616 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6a9afd191e4f40c2a679860b4257c4a7 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "6a9afd191e4f40c2a679860b4257c4a7" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6a9afd191e4f40c2a679860b4257c4a7" member_type: VOTER last_known_addr { host: "127.3.89.254" port: 38093 } } }
I20260809 15:36:10.858114 11616 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6a9afd191e4f40c2a679860b4257c4a7 [sys.catalog]: This master's current role is: LEADER
I20260809 15:36:10.858417 11617 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6a9afd191e4f40c2a679860b4257c4a7 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 6a9afd191e4f40c2a679860b4257c4a7. Latest consensus state: current_term: 1 leader_uuid: "6a9afd191e4f40c2a679860b4257c4a7" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6a9afd191e4f40c2a679860b4257c4a7" member_type: VOTER last_known_addr { host: "127.3.89.254" port: 38093 } } }
I20260809 15:36:10.858529 11617 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6a9afd191e4f40c2a679860b4257c4a7 [sys.catalog]: This master's current role is: LEADER
I20260809 15:36:10.860896 11613 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:36:10.846013 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39634 (local address 127.3.89.254:38093)
0809 15:36:10.846502 (+ 489us) server_negotiation.cc:206] Beginning negotiation
0809 15:36:10.846524 (+ 22us) server_negotiation.cc:399] Waiting for connection header
0809 15:36:10.846569 (+ 45us) server_negotiation.cc:407] Connection header received
0809 15:36:10.847505 (+ 936us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 15:36:10.847528 (+ 23us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 15:36:10.847918 (+ 390us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 15:36:10.848258 (+ 340us) server_negotiation.cc:226] Negotiated authn=SASL
0809 15:36:10.849041 (+ 783us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:36:10.850656 (+ 1615us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:36:10.851779 (+ 1123us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:36:10.852488 (+ 709us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:36:10.852657 (+ 169us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:36:10.855592 (+ 2935us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 15:36:10.855600 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 15:36:10.855620 (+ 20us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 15:36:10.855646 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0809 15:36:10.857694 (+ 2048us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 15:36:10.858226 (+ 532us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 15:36:10.858231 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 15:36:10.858237 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0809 15:36:10.858386 (+ 149us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 15:36:10.858702 (+ 316us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 15:36:10.858705 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 15:36:10.858706 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 15:36:10.859230 (+ 524us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 15:36:10.859450 (+ 220us) server_negotiation.cc:1035] Waiting for connection context
0809 15:36:10.859876 (+ 426us) server_negotiation.cc:299] Negotiation successful
0809 15:36:10.860201 (+ 325us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":339,"thread_start_us":168,"threads_started":1}
W20260809 15:36:10.862095 11631 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 6a9afd191e4f40c2a679860b4257c4a7: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260809 15:36:10.862177 11631 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260809 15:36:10.862783 11625 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 15:36:10.863754 11625 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 15:36:10.869052 11625 catalog_manager.cc:1347] Generated new cluster ID: de13e24addab45aaac7dcee9057135b2
I20260809 15:36:10.869119 11625 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 15:36:10.884900 11625 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 15:36:10.885803 11625 catalog_manager.cc:1504] Loading token signing keys...
I20260809 15:36:10.890247 11625 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 6a9afd191e4f40c2a679860b4257c4a7: Generated new TSK 0
I20260809 15:36:10.890908 11625 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for oryx/127.3.89.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.3.89.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.3.89.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786289732823074-3431-0/krb5kdc/oryx_127.3.89.193.keytab.
Entry for principal oryx/127.3.89.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786289732823074-3431-0/krb5kdc/oryx_127.3.89.193.keytab.
I20260809 15:36:10.929728 3431 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskm314YR/build/debug/bin/kudu
/tmp/dist-test-taskm314YR/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786289732823074-3431-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786289732823074-3431-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786289732823074-3431-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786289732823074-3431-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.3.89.193:0
--local_ip_for_outbound_sockets=127.3.89.193
--webserver_interface=127.3.89.193
--webserver_port=0
--tserver_master_addrs=127.3.89.254:38093
--keytab_file=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786289732823074-3431-0/krb5kdc/oryx_127.3.89.193.keytab
--principal=oryx/127.3.89.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.3.89.212:36197
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786289732823074-3431-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786289732823074-3431-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786289732823074-3431-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 15:36:11.039915 11637 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 15:36:11.040205 11637 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 15:36:11.040282 11637 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 15:36:11.044206 11637 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 15:36:11.044283 11637 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 15:36:11.044368 11637 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260809 15:36:11.044442 11637 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.3.89.193
I20260809 15:36:11.049894 11637 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.3.89.212:36197
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786289732823074-3431-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786289732823074-3431-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786289732823074-3431-0/krb5kdc/oryx_127.3.89.193.keytab
--principal=oryx/127.3.89.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.3.89.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786289732823074-3431-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.3.89.193
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.3.89.254:38093
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.11637
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.3.89.193
--log_dir=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786289732823074-3431-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 09 Aug 2026 14:46:55 UTC on ccycloud.quasar-yswkmf.root.comops.site
build id 4576540
I20260809 15:36:11.051118 11637 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 15:36:11.052438 11637 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 15:36:11.062757 11646 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 15:36:11.062906 11637 server_base.cc:1034] running on GCE node
W20260809 15:36:11.062991 11642 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 15:36:11.063024 11644 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 15:36:11.063452 11637 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 15:36:11.064069 11637 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 15:36:11.065263 11637 hybrid_clock.cc:648] HybridClock initialized: now 1786289771065238 us; error 33 us; skew 500 ppm
Aug 09 15:36:11 dist-test-slave-2w3w krb5kdc[11515](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786289771, etypes {rep=17 tkt=17 ses=17}, oryx/127.3.89.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 15:36:11.068104 11637 init.cc:377] Logged in from keytab as oryx/127.3.89.193@KRBTEST.COM (short username oryx)
I20260809 15:36:11.068908 11637 webserver.cc:460] Webserver started at http://127.3.89.193:46665/ using document root <none> and password file <none>
I20260809 15:36:11.069453 11637 fs_manager.cc:359] Metadata directory not provided
I20260809 15:36:11.069543 11637 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 15:36:11.069818 11637 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 15:36:11.071841 11637 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786289732823074-3431-0/minicluster-data/ts-0/data/instance:
uuid: "d50738272bb8411c92663d6b4be495d0"
format_stamp: "Formatted at 2026-08-09 15:36:11 on dist-test-slave-2w3w"
server_key: "16f83e9f8ece1533a79fa61706486fbb"
server_key_iv: "a587cd0c84098efb7326574a0dec68de"
server_key_version: "encryptionkey@0"
I20260809 15:36:11.072382 11637 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786289732823074-3431-0/minicluster-data/ts-0/wal/instance:
uuid: "d50738272bb8411c92663d6b4be495d0"
format_stamp: "Formatted at 2026-08-09 15:36:11 on dist-test-slave-2w3w"
server_key: "16f83e9f8ece1533a79fa61706486fbb"
server_key_iv: "a587cd0c84098efb7326574a0dec68de"
server_key_version: "encryptionkey@0"
I20260809 15:36:11.075986 11637 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.004s sys 0.000s
I20260809 15:36:11.078262 11653 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 15:36:11.079478 11637 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260809 15:36:11.079608 11637 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786289732823074-3431-0/minicluster-data/ts-0/data,/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786289732823074-3431-0/minicluster-data/ts-0/wal
uuid: "d50738272bb8411c92663d6b4be495d0"
format_stamp: "Formatted at 2026-08-09 15:36:11 on dist-test-slave-2w3w"
server_key: "16f83e9f8ece1533a79fa61706486fbb"
server_key_iv: "a587cd0c84098efb7326574a0dec68de"
server_key_version: "encryptionkey@0"
I20260809 15:36:11.079715 11637 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786289732823074-3431-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786289732823074-3431-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786289732823074-3431-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 15:36:11.095181 11637 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 15:36:11.098738 11637 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 15:36:11.098994 11637 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 15:36:11.100415 11637 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 15:36:11.100493 11637 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 15:36:11.100559 11637 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 15:36:11.100605 11637 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 09 15:36:11 dist-test-slave-2w3w krb5kdc[11515](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786289771, etypes {rep=17 tkt=17 ses=17}, oryx/127.3.89.193@KRBTEST.COM for oryx/127.3.89.254@KRBTEST.COM
I20260809 15:36:11.111712 11637 rpc_server.cc:307] RPC server started. Bound to: 127.3.89.193:42041
I20260809 15:36:11.112444 11778 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.3.89.193:42041 every 8 connection(s)
I20260809 15:36:11.112636 11637 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786289732823074-3431-0/minicluster-data/ts-0/data/info.pb
I20260809 15:36:11.114398 11613 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:36:11.101335 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.89.193:36429 (local address 127.3.89.254:38093)
0809 15:36:11.101497 (+ 162us) server_negotiation.cc:206] Beginning negotiation
0809 15:36:11.101502 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 15:36:11.102358 (+ 856us) server_negotiation.cc:407] Connection header received
0809 15:36:11.103639 (+ 1281us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 15:36:11.103645 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 15:36:11.103713 (+ 68us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 15:36:11.103779 (+ 66us) server_negotiation.cc:226] Negotiated authn=SASL
0809 15:36:11.105296 (+ 1517us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:36:11.106032 (+ 736us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:36:11.107253 (+ 1221us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:36:11.107620 (+ 367us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:36:11.107744 (+ 124us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:36:11.110631 (+ 2887us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 15:36:11.110635 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 15:36:11.110638 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 15:36:11.110667 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0809 15:36:11.112322 (+ 1655us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 15:36:11.112911 (+ 589us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 15:36:11.112914 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 15:36:11.112915 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 15:36:11.113022 (+ 107us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 15:36:11.113429 (+ 407us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 15:36:11.113435 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 15:36:11.113439 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 15:36:11.113644 (+ 205us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 15:36:11.113728 (+ 84us) server_negotiation.cc:1035] Waiting for connection context
0809 15:36:11.114170 (+ 442us) server_negotiation.cc:299] Negotiation successful
0809 15:36:11.114274 (+ 104us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":65}
I20260809 15:36:11.115231 11663 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:36:11.101731 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.89.254:38093 (local address 127.3.89.193:36429)
0809 15:36:11.102206 (+ 475us) negotiation.cc:107] Waiting for socket to connect
0809 15:36:11.102257 (+ 51us) client_negotiation.cc:174] Beginning negotiation
0809 15:36:11.103382 (+ 1125us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 15:36:11.104161 (+ 779us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 15:36:11.104167 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 15:36:11.104567 (+ 400us) client_negotiation.cc:189] Negotiated authn=SASL
0809 15:36:11.105141 (+ 574us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:36:11.105158 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:36:11.106171 (+ 1013us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:36:11.106175 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:36:11.107121 (+ 946us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:36:11.107131 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:36:11.107795 (+ 664us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:36:11.107798 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:36:11.107969 (+ 171us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:36:11.108543 (+ 574us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 15:36:11.108559 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0809 15:36:11.110428 (+ 1869us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 15:36:11.112445 (+ 2017us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 15:36:11.112450 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 15:36:11.112463 (+ 13us) client_negotiation.cc:760] calling sasl_client_step()
0809 15:36:11.112807 (+ 344us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 15:36:11.113147 (+ 340us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 15:36:11.113150 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 15:36:11.113152 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 15:36:11.113311 (+ 159us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 15:36:11.113751 (+ 440us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 15:36:11.113755 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 15:36:11.113969 (+ 214us) client_negotiation.cc:769] Sending connection context
0809 15:36:11.114120 (+ 151us) client_negotiation.cc:240] Negotiation successful
0809 15:36:11.114347 (+ 227us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":282,"thread_start_us":147,"threads_started":1}
I20260809 15:36:11.116230 3431 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskm314YR/build/debug/bin/kudu as pid 11637
I20260809 15:36:11.116365 3431 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786289732823074-3431-0/minicluster-data/ts-0/wal/instance
I20260809 15:36:11.116650 11779 heartbeater.cc:347] Connected to a master server at 127.3.89.254:38093
I20260809 15:36:11.116703 3431 external_mini_cluster.cc:1428] Setting key 3cd214b5a4e43f198db58c3d2c624591
I20260809 15:36:11.116894 11779 heartbeater.cc:464] Registering TS with master...
I20260809 15:36:11.117449 11779 heartbeater.cc:511] Master 127.3.89.254:38093 requested a full tablet report, sending...
I20260809 15:36:11.119711 11559 ts_manager.cc:194] Registered new tserver with Master: d50738272bb8411c92663d6b4be495d0 (127.3.89.193:42041)
I20260809 15:36:11.120977 11559 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.3.89.193@KRBTEST.COM'} at 127.3.89.193:36429
I20260809 15:36:11.126523 11786 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:36:11.118650 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.89.254:38093 (local address 127.3.89.193:45943)
0809 15:36:11.119000 (+ 350us) negotiation.cc:107] Waiting for socket to connect
0809 15:36:11.119014 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0809 15:36:11.119114 (+ 100us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 15:36:11.119342 (+ 228us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 15:36:11.119345 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 15:36:11.119530 (+ 185us) client_negotiation.cc:189] Negotiated authn=SASL
0809 15:36:11.119620 (+ 90us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:36:11.119625 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:36:11.120493 (+ 868us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:36:11.120500 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:36:11.121477 (+ 977us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:36:11.121489 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:36:11.122182 (+ 693us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:36:11.122186 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:36:11.122276 (+ 90us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:36:11.122747 (+ 471us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 15:36:11.122770 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0809 15:36:11.123190 (+ 420us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 15:36:11.125131 (+ 1941us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 15:36:11.125133 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 15:36:11.125135 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 15:36:11.125322 (+ 187us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 15:36:11.125616 (+ 294us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 15:36:11.125618 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 15:36:11.125620 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 15:36:11.125692 (+ 72us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 15:36:11.126136 (+ 444us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 15:36:11.126138 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 15:36:11.126207 (+ 69us) client_negotiation.cc:769] Sending connection context
0809 15:36:11.126288 (+ 81us) client_negotiation.cc:240] Negotiation successful
0809 15:36:11.126386 (+ 98us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":247,"thread_start_us":156,"threads_started":1}
I20260809 15:36:11.126678 11613 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:36:11.118788 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.89.193:45943 (local address 127.3.89.254:38093)
0809 15:36:11.118933 (+ 145us) server_negotiation.cc:206] Beginning negotiation
0809 15:36:11.118939 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 15:36:11.119090 (+ 151us) server_negotiation.cc:407] Connection header received
0809 15:36:11.119190 (+ 100us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 15:36:11.119193 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 15:36:11.119232 (+ 39us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 15:36:11.119328 (+ 96us) server_negotiation.cc:226] Negotiated authn=SASL
0809 15:36:11.119732 (+ 404us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:36:11.120321 (+ 589us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:36:11.121625 (+ 1304us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:36:11.121959 (+ 334us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:36:11.122037 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:36:11.123342 (+ 1305us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 15:36:11.123347 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 15:36:11.123351 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 15:36:11.123383 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0809 15:36:11.125006 (+ 1623us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 15:36:11.125420 (+ 414us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 15:36:11.125424 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 15:36:11.125427 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 15:36:11.125518 (+ 91us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 15:36:11.125788 (+ 270us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 15:36:11.125791 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 15:36:11.125793 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 15:36:11.126037 (+ 244us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 15:36:11.126144 (+ 107us) server_negotiation.cc:1035] Waiting for connection context
0809 15:36:11.126380 (+ 236us) server_negotiation.cc:299] Negotiation successful
0809 15:36:11.126514 (+ 134us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
WARNING: no policy specified for oryx/127.3.89.194@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.3.89.194@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.3.89.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786289732823074-3431-0/krb5kdc/oryx_127.3.89.194.keytab.
Entry for principal oryx/127.3.89.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786289732823074-3431-0/krb5kdc/oryx_127.3.89.194.keytab.
I20260809 15:36:11.146674 3431 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskm314YR/build/debug/bin/kudu
/tmp/dist-test-taskm314YR/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786289732823074-3431-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786289732823074-3431-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786289732823074-3431-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786289732823074-3431-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.3.89.194:0
--local_ip_for_outbound_sockets=127.3.89.194
--webserver_interface=127.3.89.194
--webserver_port=0
--tserver_master_addrs=127.3.89.254:38093
--keytab_file=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786289732823074-3431-0/krb5kdc/oryx_127.3.89.194.keytab
--principal=oryx/127.3.89.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.3.89.212:36197
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786289732823074-3431-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786289732823074-3431-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786289732823074-3431-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 15:36:11.256729 11790 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 15:36:11.256974 11790 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 15:36:11.257025 11790 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 15:36:11.261123 11790 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 15:36:11.261179 11790 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 15:36:11.261219 11790 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260809 15:36:11.261250 11790 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.3.89.194
I20260809 15:36:11.266281 11790 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.3.89.212:36197
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786289732823074-3431-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786289732823074-3431-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786289732823074-3431-0/krb5kdc/oryx_127.3.89.194.keytab
--principal=oryx/127.3.89.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.3.89.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786289732823074-3431-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.3.89.194
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.3.89.254:38093
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.11790
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.3.89.194
--log_dir=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786289732823074-3431-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 09 Aug 2026 14:46:55 UTC on ccycloud.quasar-yswkmf.root.comops.site
build id 4576540
I20260809 15:36:11.267279 11790 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 15:36:11.268285 11790 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 15:36:11.277531 11796 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 15:36:11.277541 11795 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 15:36:11.277541 11798 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 15:36:11.277745 11790 server_base.cc:1034] running on GCE node
I20260809 15:36:11.278184 11790 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 15:36:11.278793 11790 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 15:36:11.279959 11790 hybrid_clock.cc:648] HybridClock initialized: now 1786289771279958 us; error 53 us; skew 500 ppm
Aug 09 15:36:11 dist-test-slave-2w3w krb5kdc[11515](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786289771, etypes {rep=17 tkt=17 ses=17}, oryx/127.3.89.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 15:36:11.282649 11790 init.cc:377] Logged in from keytab as oryx/127.3.89.194@KRBTEST.COM (short username oryx)
I20260809 15:36:11.283573 11790 webserver.cc:460] Webserver started at http://127.3.89.194:45849/ using document root <none> and password file <none>
I20260809 15:36:11.284101 11790 fs_manager.cc:359] Metadata directory not provided
I20260809 15:36:11.284189 11790 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 15:36:11.284420 11790 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 15:36:11.286341 11790 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786289732823074-3431-0/minicluster-data/ts-1/data/instance:
uuid: "2b150c9788e84505a9f96579e7b19e75"
format_stamp: "Formatted at 2026-08-09 15:36:11 on dist-test-slave-2w3w"
server_key: "e1f65e50123b964c0a022df2e9855d20"
server_key_iv: "9fc28af8d9e848c7e7385eef5e5837d9"
server_key_version: "encryptionkey@0"
I20260809 15:36:11.286907 11790 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786289732823074-3431-0/minicluster-data/ts-1/wal/instance:
uuid: "2b150c9788e84505a9f96579e7b19e75"
format_stamp: "Formatted at 2026-08-09 15:36:11 on dist-test-slave-2w3w"
server_key: "e1f65e50123b964c0a022df2e9855d20"
server_key_iv: "9fc28af8d9e848c7e7385eef5e5837d9"
server_key_version: "encryptionkey@0"
I20260809 15:36:11.290385 11790 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.002s sys 0.000s
I20260809 15:36:11.292732 11805 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 15:36:11.293848 11790 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.000s sys 0.001s
I20260809 15:36:11.293987 11790 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786289732823074-3431-0/minicluster-data/ts-1/data,/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786289732823074-3431-0/minicluster-data/ts-1/wal
uuid: "2b150c9788e84505a9f96579e7b19e75"
format_stamp: "Formatted at 2026-08-09 15:36:11 on dist-test-slave-2w3w"
server_key: "e1f65e50123b964c0a022df2e9855d20"
server_key_iv: "9fc28af8d9e848c7e7385eef5e5837d9"
server_key_version: "encryptionkey@0"
I20260809 15:36:11.294093 11790 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786289732823074-3431-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786289732823074-3431-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786289732823074-3431-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 15:36:11.311185 11790 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 15:36:11.313970 11790 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 15:36:11.314169 11790 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 15:36:11.315676 11790 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 15:36:11.315747 11790 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 15:36:11.315814 11790 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 15:36:11.315874 11790 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 09 15:36:11 dist-test-slave-2w3w krb5kdc[11515](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786289771, etypes {rep=17 tkt=17 ses=17}, oryx/127.3.89.194@KRBTEST.COM for oryx/127.3.89.254@KRBTEST.COM
I20260809 15:36:11.326290 11790 rpc_server.cc:307] RPC server started. Bound to: 127.3.89.194:34141
I20260809 15:36:11.326360 11930 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.3.89.194:34141 every 8 connection(s)
I20260809 15:36:11.327224 11790 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786289732823074-3431-0/minicluster-data/ts-1/data/info.pb
I20260809 15:36:11.330155 11613 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:36:11.316417 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.89.194:42041 (local address 127.3.89.254:38093)
0809 15:36:11.316564 (+ 147us) server_negotiation.cc:206] Beginning negotiation
0809 15:36:11.316569 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 15:36:11.317504 (+ 935us) server_negotiation.cc:407] Connection header received
0809 15:36:11.318696 (+ 1192us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 15:36:11.318700 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 15:36:11.318768 (+ 68us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 15:36:11.318874 (+ 106us) server_negotiation.cc:226] Negotiated authn=SASL
0809 15:36:11.320359 (+ 1485us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:36:11.321260 (+ 901us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:36:11.323064 (+ 1804us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:36:11.323419 (+ 355us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:36:11.323532 (+ 113us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:36:11.326386 (+ 2854us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 15:36:11.326392 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 15:36:11.326395 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 15:36:11.326421 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0809 15:36:11.328165 (+ 1744us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 15:36:11.328666 (+ 501us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 15:36:11.328670 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 15:36:11.328672 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 15:36:11.328757 (+ 85us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 15:36:11.329109 (+ 352us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 15:36:11.329114 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 15:36:11.329115 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 15:36:11.329300 (+ 185us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 15:36:11.329413 (+ 113us) server_negotiation.cc:1035] Waiting for connection context
0809 15:36:11.329866 (+ 453us) server_negotiation.cc:299] Negotiation successful
0809 15:36:11.329986 (+ 120us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":55}
I20260809 15:36:11.330830 11814 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:36:11.316904 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.89.254:38093 (local address 127.3.89.194:42041)
0809 15:36:11.317340 (+ 436us) negotiation.cc:107] Waiting for socket to connect
0809 15:36:11.317389 (+ 49us) client_negotiation.cc:174] Beginning negotiation
0809 15:36:11.318465 (+ 1076us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 15:36:11.319085 (+ 620us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 15:36:11.319095 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 15:36:11.319544 (+ 449us) client_negotiation.cc:189] Negotiated authn=SASL
0809 15:36:11.320177 (+ 633us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:36:11.320196 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:36:11.321405 (+ 1209us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:36:11.321409 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:36:11.322925 (+ 1516us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:36:11.322939 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:36:11.323576 (+ 637us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:36:11.323579 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:36:11.323715 (+ 136us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:36:11.324263 (+ 548us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 15:36:11.324285 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0809 15:36:11.326229 (+ 1944us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 15:36:11.328274 (+ 2045us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 15:36:11.328278 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 15:36:11.328288 (+ 10us) client_negotiation.cc:760] calling sasl_client_step()
0809 15:36:11.328530 (+ 242us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 15:36:11.328860 (+ 330us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 15:36:11.328863 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 15:36:11.328865 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 15:36:11.329009 (+ 144us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 15:36:11.329411 (+ 402us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 15:36:11.329418 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 15:36:11.329636 (+ 218us) client_negotiation.cc:769] Sending connection context
0809 15:36:11.329820 (+ 184us) client_negotiation.cc:240] Negotiation successful
0809 15:36:11.330118 (+ 298us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":251,"thread_start_us":127,"threads_started":1}
I20260809 15:36:11.332029 11931 heartbeater.cc:347] Connected to a master server at 127.3.89.254:38093
I20260809 15:36:11.332159 11931 heartbeater.cc:464] Registering TS with master...
I20260809 15:36:11.332671 11931 heartbeater.cc:511] Master 127.3.89.254:38093 requested a full tablet report, sending...
I20260809 15:36:11.333146 3431 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskm314YR/build/debug/bin/kudu as pid 11790
I20260809 15:36:11.333292 3431 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786289732823074-3431-0/minicluster-data/ts-1/wal/instance
I20260809 15:36:11.333662 3431 external_mini_cluster.cc:1428] Setting key cbdc747a3811bc66202807d8c3af770a
I20260809 15:36:11.333777 11560 ts_manager.cc:194] Registered new tserver with Master: 2b150c9788e84505a9f96579e7b19e75 (127.3.89.194:34141)
I20260809 15:36:11.334538 11560 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.3.89.194@KRBTEST.COM'} at 127.3.89.194:42041
I20260809 15:36:11.342912 11937 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:36:11.334259 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.89.254:38093 (local address 127.3.89.194:57397)
0809 15:36:11.334727 (+ 468us) negotiation.cc:107] Waiting for socket to connect
0809 15:36:11.334746 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0809 15:36:11.334890 (+ 144us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 15:36:11.336279 (+ 1389us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 15:36:11.336281 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 15:36:11.336437 (+ 156us) client_negotiation.cc:189] Negotiated authn=SASL
0809 15:36:11.336524 (+ 87us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:36:11.336529 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:36:11.337481 (+ 952us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:36:11.337484 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:36:11.338045 (+ 561us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:36:11.338053 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:36:11.338469 (+ 416us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:36:11.338472 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:36:11.338539 (+ 67us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:36:11.339018 (+ 479us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 15:36:11.339031 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0809 15:36:11.339332 (+ 301us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 15:36:11.341416 (+ 2084us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 15:36:11.341418 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 15:36:11.341419 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0809 15:36:11.341600 (+ 181us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 15:36:11.341916 (+ 316us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 15:36:11.341918 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 15:36:11.341919 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0809 15:36:11.341991 (+ 72us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 15:36:11.342523 (+ 532us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 15:36:11.342525 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 15:36:11.342593 (+ 68us) client_negotiation.cc:769] Sending connection context
0809 15:36:11.342668 (+ 75us) client_negotiation.cc:240] Negotiation successful
0809 15:36:11.342760 (+ 92us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":383,"thread_start_us":119,"threads_started":1}
I20260809 15:36:11.343102 11613 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:36:11.334528 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.89.194:57397 (local address 127.3.89.254:38093)
0809 15:36:11.334648 (+ 120us) server_negotiation.cc:206] Beginning negotiation
0809 15:36:11.334652 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 15:36:11.334826 (+ 174us) server_negotiation.cc:407] Connection header received
0809 15:36:11.335026 (+ 200us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 15:36:11.335029 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 15:36:11.335077 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 15:36:11.335165 (+ 88us) server_negotiation.cc:226] Negotiated authn=SASL
0809 15:36:11.336643 (+ 1478us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:36:11.337341 (+ 698us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:36:11.338170 (+ 829us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:36:11.338373 (+ 203us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:36:11.338453 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:36:11.339479 (+ 1026us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 15:36:11.339485 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 15:36:11.339491 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 15:36:11.339522 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0809 15:36:11.341305 (+ 1783us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 15:36:11.341702 (+ 397us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 15:36:11.341707 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 15:36:11.341711 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 15:36:11.341812 (+ 101us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 15:36:11.342088 (+ 276us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 15:36:11.342094 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 15:36:11.342099 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0809 15:36:11.342417 (+ 318us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 15:36:11.342530 (+ 113us) server_negotiation.cc:1035] Waiting for connection context
0809 15:36:11.342759 (+ 229us) server_negotiation.cc:299] Negotiation successful
0809 15:36:11.342880 (+ 121us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":46}
WARNING: no policy specified for oryx/127.3.89.195@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.3.89.195@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.3.89.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786289732823074-3431-0/krb5kdc/oryx_127.3.89.195.keytab.
Entry for principal oryx/127.3.89.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786289732823074-3431-0/krb5kdc/oryx_127.3.89.195.keytab.
I20260809 15:36:11.364144 3431 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskm314YR/build/debug/bin/kudu
/tmp/dist-test-taskm314YR/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786289732823074-3431-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786289732823074-3431-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786289732823074-3431-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786289732823074-3431-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.3.89.195:0
--local_ip_for_outbound_sockets=127.3.89.195
--webserver_interface=127.3.89.195
--webserver_port=0
--tserver_master_addrs=127.3.89.254:38093
--keytab_file=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786289732823074-3431-0/krb5kdc/oryx_127.3.89.195.keytab
--principal=oryx/127.3.89.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.3.89.212:36197
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786289732823074-3431-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786289732823074-3431-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786289732823074-3431-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 15:36:11.474545 11940 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 15:36:11.474795 11940 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 15:36:11.474841 11940 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 15:36:11.478816 11940 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 15:36:11.478866 11940 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 15:36:11.478922 11940 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260809 15:36:11.478953 11940 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.3.89.195
I20260809 15:36:11.484274 11940 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.3.89.212:36197
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786289732823074-3431-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786289732823074-3431-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786289732823074-3431-0/krb5kdc/oryx_127.3.89.195.keytab
--principal=oryx/127.3.89.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.3.89.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786289732823074-3431-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.3.89.195
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.3.89.254:38093
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.11940
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.3.89.195
--log_dir=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786289732823074-3431-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 09 Aug 2026 14:46:55 UTC on ccycloud.quasar-yswkmf.root.comops.site
build id 4576540
I20260809 15:36:11.485190 11940 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 15:36:11.486099 11940 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 15:36:11.495324 11946 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 15:36:11.495354 11945 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 15:36:11.495437 11940 server_base.cc:1034] running on GCE node
W20260809 15:36:11.495637 11948 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 15:36:11.496007 11940 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 15:36:11.496593 11940 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 15:36:11.497769 11940 hybrid_clock.cc:648] HybridClock initialized: now 1786289771497759 us; error 28 us; skew 500 ppm
Aug 09 15:36:11 dist-test-slave-2w3w krb5kdc[11515](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786289771, etypes {rep=17 tkt=17 ses=17}, oryx/127.3.89.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 15:36:11.500643 11940 init.cc:377] Logged in from keytab as oryx/127.3.89.195@KRBTEST.COM (short username oryx)
I20260809 15:36:11.501480 11940 webserver.cc:460] Webserver started at http://127.3.89.195:43311/ using document root <none> and password file <none>
I20260809 15:36:11.502043 11940 fs_manager.cc:359] Metadata directory not provided
I20260809 15:36:11.502135 11940 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 15:36:11.502365 11940 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 15:36:11.504387 11940 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786289732823074-3431-0/minicluster-data/ts-2/data/instance:
uuid: "13f0c3e0f54b458484b5a99d9b58fc82"
format_stamp: "Formatted at 2026-08-09 15:36:11 on dist-test-slave-2w3w"
server_key: "726eaa6b347d3db04e2701beb2d18948"
server_key_iv: "e944c70196dab388f2cd48fdd8206792"
server_key_version: "encryptionkey@0"
I20260809 15:36:11.504972 11940 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786289732823074-3431-0/minicluster-data/ts-2/wal/instance:
uuid: "13f0c3e0f54b458484b5a99d9b58fc82"
format_stamp: "Formatted at 2026-08-09 15:36:11 on dist-test-slave-2w3w"
server_key: "726eaa6b347d3db04e2701beb2d18948"
server_key_iv: "e944c70196dab388f2cd48fdd8206792"
server_key_version: "encryptionkey@0"
I20260809 15:36:11.508590 11940 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.000s sys 0.005s
I20260809 15:36:11.511155 11955 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 15:36:11.512353 11940 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.001s
I20260809 15:36:11.512497 11940 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786289732823074-3431-0/minicluster-data/ts-2/data,/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786289732823074-3431-0/minicluster-data/ts-2/wal
uuid: "13f0c3e0f54b458484b5a99d9b58fc82"
format_stamp: "Formatted at 2026-08-09 15:36:11 on dist-test-slave-2w3w"
server_key: "726eaa6b347d3db04e2701beb2d18948"
server_key_iv: "e944c70196dab388f2cd48fdd8206792"
server_key_version: "encryptionkey@0"
I20260809 15:36:11.512609 11940 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786289732823074-3431-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786289732823074-3431-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786289732823074-3431-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 15:36:11.525115 11940 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 15:36:11.528210 11940 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 15:36:11.528404 11940 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 15:36:11.529736 11940 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 15:36:11.529811 11940 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 15:36:11.529874 11940 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 15:36:11.529925 11940 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 09 15:36:11 dist-test-slave-2w3w krb5kdc[11515](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786289771, etypes {rep=17 tkt=17 ses=17}, oryx/127.3.89.195@KRBTEST.COM for oryx/127.3.89.254@KRBTEST.COM
I20260809 15:36:11.541267 11940 rpc_server.cc:307] RPC server started. Bound to: 127.3.89.195:43403
I20260809 15:36:11.541388 12086 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.3.89.195:43403 every 8 connection(s)
I20260809 15:36:11.542150 11940 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786289732823074-3431-0/minicluster-data/ts-2/data/info.pb
I20260809 15:36:11.544028 11613 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:36:11.530608 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.89.195:35743 (local address 127.3.89.254:38093)
0809 15:36:11.530735 (+ 127us) server_negotiation.cc:206] Beginning negotiation
0809 15:36:11.530738 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0809 15:36:11.531806 (+ 1068us) server_negotiation.cc:407] Connection header received
0809 15:36:11.532799 (+ 993us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 15:36:11.532803 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 15:36:11.532867 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 15:36:11.532958 (+ 91us) server_negotiation.cc:226] Negotiated authn=SASL
0809 15:36:11.534565 (+ 1607us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:36:11.535470 (+ 905us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:36:11.537310 (+ 1840us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:36:11.537678 (+ 368us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:36:11.537793 (+ 115us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:36:11.540230 (+ 2437us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 15:36:11.540234 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 15:36:11.540237 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 15:36:11.540262 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0809 15:36:11.541919 (+ 1657us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 15:36:11.542497 (+ 578us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 15:36:11.542500 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 15:36:11.542502 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 15:36:11.542622 (+ 120us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 15:36:11.543024 (+ 402us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 15:36:11.543027 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 15:36:11.543029 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 15:36:11.543249 (+ 220us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 15:36:11.543342 (+ 93us) server_negotiation.cc:1035] Waiting for connection context
0809 15:36:11.543809 (+ 467us) server_negotiation.cc:299] Negotiation successful
0809 15:36:11.543911 (+ 102us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":48}
I20260809 15:36:11.544724 11967 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:36:11.531151 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.89.254:38093 (local address 127.3.89.195:35743)
0809 15:36:11.531630 (+ 479us) negotiation.cc:107] Waiting for socket to connect
0809 15:36:11.531672 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0809 15:36:11.532586 (+ 914us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 15:36:11.533152 (+ 566us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 15:36:11.533163 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 15:36:11.533718 (+ 555us) client_negotiation.cc:189] Negotiated authn=SASL
0809 15:36:11.534382 (+ 664us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:36:11.534407 (+ 25us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:36:11.535590 (+ 1183us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:36:11.535594 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:36:11.537149 (+ 1555us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:36:11.537164 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:36:11.537841 (+ 677us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:36:11.537846 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:36:11.538021 (+ 175us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:36:11.538542 (+ 521us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 15:36:11.538557 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0809 15:36:11.540062 (+ 1505us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 15:36:11.542040 (+ 1978us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 15:36:11.542046 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 15:36:11.542066 (+ 20us) client_negotiation.cc:760] calling sasl_client_step()
0809 15:36:11.542387 (+ 321us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 15:36:11.542725 (+ 338us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 15:36:11.542729 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 15:36:11.542733 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0809 15:36:11.542889 (+ 156us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 15:36:11.543349 (+ 460us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 15:36:11.543353 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 15:36:11.543554 (+ 201us) client_negotiation.cc:769] Sending connection context
0809 15:36:11.543693 (+ 139us) client_negotiation.cc:240] Negotiation successful
0809 15:36:11.543917 (+ 224us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":288,"thread_start_us":138,"threads_started":1}
I20260809 15:36:11.545907 12087 heartbeater.cc:347] Connected to a master server at 127.3.89.254:38093
I20260809 15:36:11.546080 12087 heartbeater.cc:464] Registering TS with master...
I20260809 15:36:11.546550 12087 heartbeater.cc:511] Master 127.3.89.254:38093 requested a full tablet report, sending...
I20260809 15:36:11.547570 11559 ts_manager.cc:194] Registered new tserver with Master: 13f0c3e0f54b458484b5a99d9b58fc82 (127.3.89.195:43403)
I20260809 15:36:11.548269 11559 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.3.89.195@KRBTEST.COM'} at 127.3.89.195:35743
I20260809 15:36:11.550745 3431 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskm314YR/build/debug/bin/kudu as pid 11940
I20260809 15:36:11.550910 3431 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786289732823074-3431-0/minicluster-data/ts-2/wal/instance
I20260809 15:36:11.551295 3431 external_mini_cluster.cc:1428] Setting key 584480411e57179a640d2b9498fba362
I20260809 15:36:11.553433 3431 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260809 15:36:11.555341 12093 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:36:11.548113 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.89.254:38093 (local address 127.3.89.195:59301)
0809 15:36:11.548378 (+ 265us) negotiation.cc:107] Waiting for socket to connect
0809 15:36:11.548395 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0809 15:36:11.548482 (+ 87us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 15:36:11.548763 (+ 281us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 15:36:11.548766 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 15:36:11.548979 (+ 213us) client_negotiation.cc:189] Negotiated authn=SASL
0809 15:36:11.549070 (+ 91us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:36:11.549075 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:36:11.549808 (+ 733us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:36:11.549811 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:36:11.550374 (+ 563us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:36:11.550381 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:36:11.550873 (+ 492us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:36:11.550877 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:36:11.550999 (+ 122us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:36:11.551484 (+ 485us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 15:36:11.551496 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0809 15:36:11.551917 (+ 421us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 15:36:11.553800 (+ 1883us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 15:36:11.553804 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 15:36:11.553806 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 15:36:11.554002 (+ 196us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 15:36:11.554287 (+ 285us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 15:36:11.554292 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 15:36:11.554298 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0809 15:36:11.554385 (+ 87us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 15:36:11.554766 (+ 381us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 15:36:11.554776 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 15:36:11.554877 (+ 101us) client_negotiation.cc:769] Sending connection context
0809 15:36:11.555014 (+ 137us) client_negotiation.cc:240] Negotiation successful
0809 15:36:11.555158 (+ 144us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":181,"thread_start_us":113,"threads_started":1}
I20260809 15:36:11.555338 11613 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:36:11.548464 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.89.195:59301 (local address 127.3.89.254:38093)
0809 15:36:11.548575 (+ 111us) server_negotiation.cc:206] Beginning negotiation
0809 15:36:11.548579 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 15:36:11.548589 (+ 10us) server_negotiation.cc:407] Connection header received
0809 15:36:11.548621 (+ 32us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 15:36:11.548630 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 15:36:11.548661 (+ 31us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 15:36:11.548746 (+ 85us) server_negotiation.cc:226] Negotiated authn=SASL
0809 15:36:11.549184 (+ 438us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:36:11.549674 (+ 490us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:36:11.550510 (+ 836us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:36:11.550757 (+ 247us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:36:11.550868 (+ 111us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:36:11.552047 (+ 1179us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 15:36:11.552050 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 15:36:11.552053 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 15:36:11.552076 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0809 15:36:11.553692 (+ 1616us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 15:36:11.554107 (+ 415us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 15:36:11.554111 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 15:36:11.554112 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 15:36:11.554196 (+ 84us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 15:36:11.554478 (+ 282us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 15:36:11.554482 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 15:36:11.554484 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 15:36:11.554657 (+ 173us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 15:36:11.554775 (+ 118us) server_negotiation.cc:1035] Waiting for connection context
0809 15:36:11.555091 (+ 316us) server_negotiation.cc:299] Negotiation successful
0809 15:36:11.555196 (+ 105us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":44}
Aug 09 15:36:11 dist-test-slave-2w3w krb5kdc[11515](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786289771, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Aug 09 15:36:11 dist-test-slave-2w3w krb5kdc[11515](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, test-user@KRBTEST.COM for kudu/127.3.89.254@KRBTEST.COM, Server not found in Kerberos database
Aug 09 15:36:11 dist-test-slave-2w3w krb5kdc[11515](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, test-user@KRBTEST.COM for kudu/127.3.89.254@KRBTEST.COM, Server not found in Kerberos database
W20260809 15:36:11.578605 11613 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 15:36:11.572618 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39646 (local address 127.3.89.254:38093)
0809 15:36:11.572758 (+ 140us) server_negotiation.cc:206] Beginning negotiation
0809 15:36:11.572761 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0809 15:36:11.572777 (+ 16us) server_negotiation.cc:407] Connection header received
0809 15:36:11.572817 (+ 40us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 15:36:11.572820 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 15:36:11.572859 (+ 39us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 15:36:11.572932 (+ 73us) server_negotiation.cc:226] Negotiated authn=SASL
0809 15:36:11.573598 (+ 666us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:36:11.574116 (+ 518us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:36:11.574942 (+ 826us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:36:11.575217 (+ 275us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:36:11.575324 (+ 107us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:36:11.578419 (+ 3095us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:39646: BlockingRecv error: recv got EOF from 127.0.0.1:39646 (error 108)
Metrics: {"server-negotiator.queue_time_us":55}
Aug 09 15:36:11 dist-test-slave-2w3w krb5kdc[11515](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786289771, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for oryx/127.3.89.254@KRBTEST.COM
I20260809 15:36:11.588214 11613 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:36:11.580074 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39660 (local address 127.3.89.254:38093)
0809 15:36:11.580210 (+ 136us) server_negotiation.cc:206] Beginning negotiation
0809 15:36:11.580214 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 15:36:11.580340 (+ 126us) server_negotiation.cc:407] Connection header received
0809 15:36:11.580504 (+ 164us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 15:36:11.580507 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 15:36:11.580544 (+ 37us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 15:36:11.580627 (+ 83us) server_negotiation.cc:226] Negotiated authn=SASL
0809 15:36:11.581219 (+ 592us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:36:11.581733 (+ 514us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:36:11.582538 (+ 805us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:36:11.582749 (+ 211us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:36:11.582854 (+ 105us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:36:11.585277 (+ 2423us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 15:36:11.585280 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 15:36:11.585282 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 15:36:11.585304 (+ 22us) server_negotiation.cc:922] calling sasl_server_start()
0809 15:36:11.586571 (+ 1267us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 15:36:11.587004 (+ 433us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 15:36:11.587010 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 15:36:11.587016 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0809 15:36:11.587093 (+ 77us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 15:36:11.587370 (+ 277us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 15:36:11.587372 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 15:36:11.587374 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 15:36:11.587565 (+ 191us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 15:36:11.587670 (+ 105us) server_negotiation.cc:1035] Waiting for connection context
0809 15:36:11.587923 (+ 253us) server_negotiation.cc:299] Negotiation successful
0809 15:36:11.588043 (+ 120us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
I20260809 15:36:11.591153 11559 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:39660:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260809 15:36:11.594128 11559 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260809 15:36:11.609052 12111 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:36:11.602729 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34258 (local address 127.3.89.194:34141)
0809 15:36:11.603654 (+ 925us) server_negotiation.cc:206] Beginning negotiation
0809 15:36:11.603666 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0809 15:36:11.603941 (+ 275us) server_negotiation.cc:407] Connection header received
0809 15:36:11.604028 (+ 87us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 15:36:11.604033 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 15:36:11.604224 (+ 191us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 15:36:11.604374 (+ 150us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 15:36:11.605255 (+ 881us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:36:11.606007 (+ 752us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:36:11.607668 (+ 1661us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:36:11.608435 (+ 767us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:36:11.608565 (+ 130us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:36:11.608729 (+ 164us) server_negotiation.cc:1035] Waiting for connection context
0809 15:36:11.608795 (+ 66us) server_negotiation.cc:299] Negotiation successful
0809 15:36:11.608889 (+ 94us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":812,"thread_start_us":162,"threads_started":1}
I20260809 15:36:11.609059 12110 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:36:11.602580 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.89.194:34141 (local address 127.0.0.1:34258)
0809 15:36:11.603591 (+ 1011us) negotiation.cc:107] Waiting for socket to connect
0809 15:36:11.603643 (+ 52us) client_negotiation.cc:174] Beginning negotiation
0809 15:36:11.603795 (+ 152us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 15:36:11.604382 (+ 587us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 15:36:11.604388 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 15:36:11.604407 (+ 19us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 15:36:11.604556 (+ 149us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:36:11.604563 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:36:11.606145 (+ 1582us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:36:11.606150 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:36:11.607543 (+ 1393us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:36:11.607553 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:36:11.608569 (+ 1016us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:36:11.608572 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:36:11.608649 (+ 77us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:36:11.608679 (+ 30us) client_negotiation.cc:769] Sending connection context
0809 15:36:11.608756 (+ 77us) client_negotiation.cc:240] Negotiation successful
0809 15:36:11.608897 (+ 141us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":637,"mutex_wait_us":2,"thread_start_us":93,"threads_started":1}
I20260809 15:36:11.609221 12109 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:36:11.602671 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.89.193:42041 (local address 127.0.0.1:43584)
0809 15:36:11.604274 (+ 1603us) negotiation.cc:107] Waiting for socket to connect
0809 15:36:11.604284 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0809 15:36:11.604393 (+ 109us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 15:36:11.604810 (+ 417us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 15:36:11.604812 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 15:36:11.604825 (+ 13us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 15:36:11.604900 (+ 75us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:36:11.604904 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:36:11.606319 (+ 1415us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:36:11.606322 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:36:11.607780 (+ 1458us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:36:11.607792 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:36:11.608852 (+ 1060us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:36:11.608855 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:36:11.608945 (+ 90us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:36:11.608965 (+ 20us) client_negotiation.cc:769] Sending connection context
0809 15:36:11.609035 (+ 70us) client_negotiation.cc:240] Negotiation successful
0809 15:36:11.609103 (+ 68us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1551,"mutex_wait_us":62,"thread_start_us":79,"threads_started":1}
I20260809 15:36:11.609776 12114 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:36:11.603594 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43584 (local address 127.3.89.193:42041)
0809 15:36:11.604131 (+ 537us) server_negotiation.cc:206] Beginning negotiation
0809 15:36:11.604148 (+ 17us) server_negotiation.cc:399] Waiting for connection header
0809 15:36:11.604409 (+ 261us) server_negotiation.cc:407] Connection header received
0809 15:36:11.604491 (+ 82us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 15:36:11.604499 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 15:36:11.604678 (+ 179us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 15:36:11.604830 (+ 152us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 15:36:11.604978 (+ 148us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:36:11.606168 (+ 1190us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:36:11.607914 (+ 1746us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:36:11.608679 (+ 765us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:36:11.608793 (+ 114us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:36:11.608938 (+ 145us) server_negotiation.cc:1035] Waiting for connection context
0809 15:36:11.609517 (+ 579us) server_negotiation.cc:299] Negotiation successful
0809 15:36:11.609593 (+ 76us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":419,"thread_start_us":114,"threads_started":1}
I20260809 15:36:11.610745 12112 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:36:11.603263 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.89.195:43403 (local address 127.0.0.1:45132)
0809 15:36:11.604567 (+ 1304us) negotiation.cc:107] Waiting for socket to connect
0809 15:36:11.604580 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0809 15:36:11.604667 (+ 87us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 15:36:11.605081 (+ 414us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 15:36:11.605083 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 15:36:11.605095 (+ 12us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 15:36:11.605148 (+ 53us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:36:11.605152 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:36:11.606432 (+ 1280us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:36:11.606435 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:36:11.607780 (+ 1345us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:36:11.607792 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:36:11.610351 (+ 2559us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:36:11.610356 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:36:11.610445 (+ 89us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:36:11.610468 (+ 23us) client_negotiation.cc:769] Sending connection context
0809 15:36:11.610551 (+ 83us) client_negotiation.cc:240] Negotiation successful
0809 15:36:11.610621 (+ 70us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1249,"mutex_wait_us":92,"thread_start_us":102,"threads_started":1}
I20260809 15:36:11.610841 12113 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:36:11.603424 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45132 (local address 127.3.89.195:43403)
0809 15:36:11.603858 (+ 434us) server_negotiation.cc:206] Beginning negotiation
0809 15:36:11.603866 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0809 15:36:11.604674 (+ 808us) server_negotiation.cc:407] Connection header received
0809 15:36:11.604773 (+ 99us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 15:36:11.604780 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 15:36:11.604953 (+ 173us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 15:36:11.605114 (+ 161us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 15:36:11.605243 (+ 129us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:36:11.606315 (+ 1072us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:36:11.609017 (+ 2702us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:36:11.610033 (+ 1016us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:36:11.610137 (+ 104us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:36:11.610280 (+ 143us) server_negotiation.cc:1035] Waiting for connection context
0809 15:36:11.610608 (+ 328us) server_negotiation.cc:299] Negotiation successful
0809 15:36:11.610705 (+ 97us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":346,"thread_start_us":109,"threads_started":1}
I20260809 15:36:11.612382 11862 tablet_service.cc:1467] Processing CreateTablet for tablet 9c10a3fa1a7243e4afc60112863d1d8e (DEFAULT_TABLE table=test-table [id=bde567cc8ba8435d86a7f918ed91346b]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 15:36:11.612790 11707 tablet_service.cc:1467] Processing CreateTablet for tablet 9c10a3fa1a7243e4afc60112863d1d8e (DEFAULT_TABLE table=test-table [id=bde567cc8ba8435d86a7f918ed91346b]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 15:36:11.613344 11862 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 9c10a3fa1a7243e4afc60112863d1d8e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 15:36:11.613317 12016 tablet_service.cc:1467] Processing CreateTablet for tablet 9c10a3fa1a7243e4afc60112863d1d8e (DEFAULT_TABLE table=test-table [id=bde567cc8ba8435d86a7f918ed91346b]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 15:36:11.613780 11707 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 9c10a3fa1a7243e4afc60112863d1d8e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 15:36:11.614194 12016 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 9c10a3fa1a7243e4afc60112863d1d8e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 15:36:11.620596 12115 tablet_bootstrap.cc:492] T 9c10a3fa1a7243e4afc60112863d1d8e P 2b150c9788e84505a9f96579e7b19e75: Bootstrap starting.
I20260809 15:36:11.621969 12117 tablet_bootstrap.cc:492] T 9c10a3fa1a7243e4afc60112863d1d8e P d50738272bb8411c92663d6b4be495d0: Bootstrap starting.
I20260809 15:36:11.621976 12116 tablet_bootstrap.cc:492] T 9c10a3fa1a7243e4afc60112863d1d8e P 13f0c3e0f54b458484b5a99d9b58fc82: Bootstrap starting.
I20260809 15:36:11.622676 12115 tablet_bootstrap.cc:654] T 9c10a3fa1a7243e4afc60112863d1d8e P 2b150c9788e84505a9f96579e7b19e75: Neither blocks nor log segments found. Creating new log.
I20260809 15:36:11.623447 12115 log.cc:824] T 9c10a3fa1a7243e4afc60112863d1d8e P 2b150c9788e84505a9f96579e7b19e75: Log is configured to *not* fsync() on all Append() calls
I20260809 15:36:11.624097 12116 tablet_bootstrap.cc:654] T 9c10a3fa1a7243e4afc60112863d1d8e P 13f0c3e0f54b458484b5a99d9b58fc82: Neither blocks nor log segments found. Creating new log.
I20260809 15:36:11.624356 12117 tablet_bootstrap.cc:654] T 9c10a3fa1a7243e4afc60112863d1d8e P d50738272bb8411c92663d6b4be495d0: Neither blocks nor log segments found. Creating new log.
I20260809 15:36:11.624917 12116 log.cc:824] T 9c10a3fa1a7243e4afc60112863d1d8e P 13f0c3e0f54b458484b5a99d9b58fc82: Log is configured to *not* fsync() on all Append() calls
I20260809 15:36:11.625137 12117 log.cc:824] T 9c10a3fa1a7243e4afc60112863d1d8e P d50738272bb8411c92663d6b4be495d0: Log is configured to *not* fsync() on all Append() calls
I20260809 15:36:11.626718 12116 tablet_bootstrap.cc:492] T 9c10a3fa1a7243e4afc60112863d1d8e P 13f0c3e0f54b458484b5a99d9b58fc82: No bootstrap required, opened a new log
I20260809 15:36:11.626758 12117 tablet_bootstrap.cc:492] T 9c10a3fa1a7243e4afc60112863d1d8e P d50738272bb8411c92663d6b4be495d0: No bootstrap required, opened a new log
I20260809 15:36:11.626921 12116 ts_tablet_manager.cc:1397] T 9c10a3fa1a7243e4afc60112863d1d8e P 13f0c3e0f54b458484b5a99d9b58fc82: Time spent bootstrapping tablet: real 0.005s user 0.003s sys 0.000s
I20260809 15:36:11.626932 12117 ts_tablet_manager.cc:1397] T 9c10a3fa1a7243e4afc60112863d1d8e P d50738272bb8411c92663d6b4be495d0: Time spent bootstrapping tablet: real 0.005s user 0.004s sys 0.000s
I20260809 15:36:11.629603 12117 raft_consensus.cc:348] T 9c10a3fa1a7243e4afc60112863d1d8e P d50738272bb8411c92663d6b4be495d0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2b150c9788e84505a9f96579e7b19e75" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 34141 } } peers { permanent_uuid: "d50738272bb8411c92663d6b4be495d0" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 42041 } } peers { permanent_uuid: "13f0c3e0f54b458484b5a99d9b58fc82" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 43403 } }
I20260809 15:36:11.629673 12116 raft_consensus.cc:348] T 9c10a3fa1a7243e4afc60112863d1d8e P 13f0c3e0f54b458484b5a99d9b58fc82 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2b150c9788e84505a9f96579e7b19e75" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 34141 } } peers { permanent_uuid: "d50738272bb8411c92663d6b4be495d0" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 42041 } } peers { permanent_uuid: "13f0c3e0f54b458484b5a99d9b58fc82" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 43403 } }
I20260809 15:36:11.629842 12117 raft_consensus.cc:374] T 9c10a3fa1a7243e4afc60112863d1d8e P d50738272bb8411c92663d6b4be495d0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 15:36:11.629904 12117 raft_consensus.cc:729] T 9c10a3fa1a7243e4afc60112863d1d8e P d50738272bb8411c92663d6b4be495d0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d50738272bb8411c92663d6b4be495d0, State: Initialized, Role: FOLLOWER
I20260809 15:36:11.629936 12116 raft_consensus.cc:374] T 9c10a3fa1a7243e4afc60112863d1d8e P 13f0c3e0f54b458484b5a99d9b58fc82 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 15:36:11.630019 12116 raft_consensus.cc:729] T 9c10a3fa1a7243e4afc60112863d1d8e P 13f0c3e0f54b458484b5a99d9b58fc82 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 13f0c3e0f54b458484b5a99d9b58fc82, State: Initialized, Role: FOLLOWER
I20260809 15:36:11.630363 12117 consensus_queue.cc:260] T 9c10a3fa1a7243e4afc60112863d1d8e P d50738272bb8411c92663d6b4be495d0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2b150c9788e84505a9f96579e7b19e75" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 34141 } } peers { permanent_uuid: "d50738272bb8411c92663d6b4be495d0" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 42041 } } peers { permanent_uuid: "13f0c3e0f54b458484b5a99d9b58fc82" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 43403 } }
I20260809 15:36:11.630419 12116 consensus_queue.cc:260] T 9c10a3fa1a7243e4afc60112863d1d8e P 13f0c3e0f54b458484b5a99d9b58fc82 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2b150c9788e84505a9f96579e7b19e75" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 34141 } } peers { permanent_uuid: "d50738272bb8411c92663d6b4be495d0" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 42041 } } peers { permanent_uuid: "13f0c3e0f54b458484b5a99d9b58fc82" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 43403 } }
I20260809 15:36:11.630815 12115 tablet_bootstrap.cc:492] T 9c10a3fa1a7243e4afc60112863d1d8e P 2b150c9788e84505a9f96579e7b19e75: No bootstrap required, opened a new log
I20260809 15:36:11.631035 12115 ts_tablet_manager.cc:1397] T 9c10a3fa1a7243e4afc60112863d1d8e P 2b150c9788e84505a9f96579e7b19e75: Time spent bootstrapping tablet: real 0.011s user 0.005s sys 0.000s
I20260809 15:36:11.631389 12116 ts_tablet_manager.cc:1428] T 9c10a3fa1a7243e4afc60112863d1d8e P 13f0c3e0f54b458484b5a99d9b58fc82: Time spent starting tablet: real 0.004s user 0.001s sys 0.003s
I20260809 15:36:11.631405 12117 ts_tablet_manager.cc:1428] T 9c10a3fa1a7243e4afc60112863d1d8e P d50738272bb8411c92663d6b4be495d0: Time spent starting tablet: real 0.004s user 0.004s sys 0.000s
I20260809 15:36:11.631743 11779 heartbeater.cc:503] Master 127.3.89.254:38093 was elected leader, sending a full tablet report...
I20260809 15:36:11.631771 12087 heartbeater.cc:503] Master 127.3.89.254:38093 was elected leader, sending a full tablet report...
I20260809 15:36:11.634523 12115 raft_consensus.cc:348] T 9c10a3fa1a7243e4afc60112863d1d8e P 2b150c9788e84505a9f96579e7b19e75 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2b150c9788e84505a9f96579e7b19e75" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 34141 } } peers { permanent_uuid: "d50738272bb8411c92663d6b4be495d0" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 42041 } } peers { permanent_uuid: "13f0c3e0f54b458484b5a99d9b58fc82" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 43403 } }
I20260809 15:36:11.634795 12115 raft_consensus.cc:374] T 9c10a3fa1a7243e4afc60112863d1d8e P 2b150c9788e84505a9f96579e7b19e75 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 15:36:11.634861 12115 raft_consensus.cc:729] T 9c10a3fa1a7243e4afc60112863d1d8e P 2b150c9788e84505a9f96579e7b19e75 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2b150c9788e84505a9f96579e7b19e75, State: Initialized, Role: FOLLOWER
I20260809 15:36:11.635294 12115 consensus_queue.cc:260] T 9c10a3fa1a7243e4afc60112863d1d8e P 2b150c9788e84505a9f96579e7b19e75 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2b150c9788e84505a9f96579e7b19e75" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 34141 } } peers { permanent_uuid: "d50738272bb8411c92663d6b4be495d0" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 42041 } } peers { permanent_uuid: "13f0c3e0f54b458484b5a99d9b58fc82" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 43403 } }
I20260809 15:36:11.636138 11931 heartbeater.cc:503] Master 127.3.89.254:38093 was elected leader, sending a full tablet report...
I20260809 15:36:11.636246 12115 ts_tablet_manager.cc:1428] T 9c10a3fa1a7243e4afc60112863d1d8e P 2b150c9788e84505a9f96579e7b19e75: Time spent starting tablet: real 0.005s user 0.004s sys 0.000s
I20260809 15:36:11.675717 12122 raft_consensus.cc:482] T 9c10a3fa1a7243e4afc60112863d1d8e P 13f0c3e0f54b458484b5a99d9b58fc82 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 15:36:11.675943 12122 raft_consensus.cc:504] T 9c10a3fa1a7243e4afc60112863d1d8e P 13f0c3e0f54b458484b5a99d9b58fc82 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2b150c9788e84505a9f96579e7b19e75" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 34141 } } peers { permanent_uuid: "d50738272bb8411c92663d6b4be495d0" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 42041 } } peers { permanent_uuid: "13f0c3e0f54b458484b5a99d9b58fc82" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 43403 } }
I20260809 15:36:11.677408 12122 leader_election.cc:288] T 9c10a3fa1a7243e4afc60112863d1d8e P 13f0c3e0f54b458484b5a99d9b58fc82 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 2b150c9788e84505a9f96579e7b19e75 (127.3.89.194:34141), d50738272bb8411c92663d6b4be495d0 (127.3.89.193:42041)
I20260809 15:36:11.681053 11967 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:36:11.677267 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.89.193:42041 (local address 127.3.89.195:50331)
0809 15:36:11.677418 (+ 151us) negotiation.cc:107] Waiting for socket to connect
0809 15:36:11.677434 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0809 15:36:11.677530 (+ 96us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 15:36:11.677985 (+ 455us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 15:36:11.677988 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 15:36:11.678006 (+ 18us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 15:36:11.678125 (+ 119us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:36:11.678133 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:36:11.679003 (+ 870us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:36:11.679007 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:36:11.679956 (+ 949us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:36:11.679972 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:36:11.680613 (+ 641us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:36:11.680617 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:36:11.680723 (+ 106us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:36:11.680745 (+ 22us) client_negotiation.cc:769] Sending connection context
0809 15:36:11.680837 (+ 92us) client_negotiation.cc:240] Negotiation successful
0809 15:36:11.680912 (+ 75us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":38}
I20260809 15:36:11.681053 12114 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:36:11.677487 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.89.195:50331 (local address 127.3.89.193:42041)
0809 15:36:11.677622 (+ 135us) server_negotiation.cc:206] Beginning negotiation
0809 15:36:11.677626 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 15:36:11.677639 (+ 13us) server_negotiation.cc:407] Connection header received
0809 15:36:11.677689 (+ 50us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 15:36:11.677692 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 15:36:11.677737 (+ 45us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 15:36:11.677811 (+ 74us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 15:36:11.678254 (+ 443us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:36:11.678868 (+ 614us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:36:11.680084 (+ 1216us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:36:11.680481 (+ 397us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:36:11.680624 (+ 143us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:36:11.680682 (+ 58us) server_negotiation.cc:1035] Waiting for connection context
0809 15:36:11.680865 (+ 183us) server_negotiation.cc:299] Negotiation successful
0809 15:36:11.680936 (+ 71us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":56}
I20260809 15:36:11.681666 11729 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9c10a3fa1a7243e4afc60112863d1d8e" candidate_uuid: "13f0c3e0f54b458484b5a99d9b58fc82" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d50738272bb8411c92663d6b4be495d0" is_pre_election: true
I20260809 15:36:11.681758 12111 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:36:11.677478 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.89.195:39959 (local address 127.3.89.194:34141)
0809 15:36:11.677602 (+ 124us) server_negotiation.cc:206] Beginning negotiation
0809 15:36:11.677608 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 15:36:11.678042 (+ 434us) server_negotiation.cc:407] Connection header received
0809 15:36:11.678194 (+ 152us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 15:36:11.678198 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 15:36:11.678251 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 15:36:11.678333 (+ 82us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 15:36:11.678628 (+ 295us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:36:11.679200 (+ 572us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:36:11.680695 (+ 1495us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:36:11.681086 (+ 391us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:36:11.681231 (+ 145us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:36:11.681284 (+ 53us) server_negotiation.cc:1035] Waiting for connection context
0809 15:36:11.681587 (+ 303us) server_negotiation.cc:299] Negotiation successful
0809 15:36:11.681654 (+ 67us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
I20260809 15:36:11.681775 12124 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:36:11.677267 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.89.194:34141 (local address 127.3.89.195:39959)
0809 15:36:11.677936 (+ 669us) negotiation.cc:107] Waiting for socket to connect
0809 15:36:11.677951 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0809 15:36:11.678083 (+ 132us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 15:36:11.678344 (+ 261us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 15:36:11.678347 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 15:36:11.678362 (+ 15us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 15:36:11.678489 (+ 127us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:36:11.678494 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:36:11.679307 (+ 813us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:36:11.679311 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:36:11.680573 (+ 1262us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:36:11.680582 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:36:11.681353 (+ 771us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:36:11.681357 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:36:11.681462 (+ 105us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:36:11.681479 (+ 17us) client_negotiation.cc:769] Sending connection context
0809 15:36:11.681569 (+ 90us) client_negotiation.cc:240] Negotiation successful
0809 15:36:11.681634 (+ 65us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":598,"mutex_wait_us":356,"thread_start_us":115,"threads_started":1}
I20260809 15:36:11.682003 11729 raft_consensus.cc:2454] T 9c10a3fa1a7243e4afc60112863d1d8e P d50738272bb8411c92663d6b4be495d0 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 13f0c3e0f54b458484b5a99d9b58fc82 in term 0.
I20260809 15:36:11.682325 11882 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9c10a3fa1a7243e4afc60112863d1d8e" candidate_uuid: "13f0c3e0f54b458484b5a99d9b58fc82" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2b150c9788e84505a9f96579e7b19e75" is_pre_election: true
I20260809 15:36:11.682518 11961 leader_election.cc:302] T 9c10a3fa1a7243e4afc60112863d1d8e P 13f0c3e0f54b458484b5a99d9b58fc82 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 13f0c3e0f54b458484b5a99d9b58fc82, d50738272bb8411c92663d6b4be495d0; no voters:
I20260809 15:36:11.682587 11882 raft_consensus.cc:2454] T 9c10a3fa1a7243e4afc60112863d1d8e P 2b150c9788e84505a9f96579e7b19e75 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 13f0c3e0f54b458484b5a99d9b58fc82 in term 0.
I20260809 15:36:11.682825 12122 raft_consensus.cc:2781] T 9c10a3fa1a7243e4afc60112863d1d8e P 13f0c3e0f54b458484b5a99d9b58fc82 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260809 15:36:11.682924 12122 raft_consensus.cc:482] T 9c10a3fa1a7243e4afc60112863d1d8e P 13f0c3e0f54b458484b5a99d9b58fc82 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 15:36:11.683007 12122 raft_consensus.cc:3037] T 9c10a3fa1a7243e4afc60112863d1d8e P 13f0c3e0f54b458484b5a99d9b58fc82 [term 0 FOLLOWER]: Advancing to term 1
I20260809 15:36:11.684116 12122 raft_consensus.cc:504] T 9c10a3fa1a7243e4afc60112863d1d8e P 13f0c3e0f54b458484b5a99d9b58fc82 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2b150c9788e84505a9f96579e7b19e75" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 34141 } } peers { permanent_uuid: "d50738272bb8411c92663d6b4be495d0" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 42041 } } peers { permanent_uuid: "13f0c3e0f54b458484b5a99d9b58fc82" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 43403 } }
I20260809 15:36:11.684453 12122 leader_election.cc:288] T 9c10a3fa1a7243e4afc60112863d1d8e P 13f0c3e0f54b458484b5a99d9b58fc82 [CANDIDATE]: Term 1 election: Requested vote from peers 2b150c9788e84505a9f96579e7b19e75 (127.3.89.194:34141), d50738272bb8411c92663d6b4be495d0 (127.3.89.193:42041)
I20260809 15:36:11.684747 11882 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9c10a3fa1a7243e4afc60112863d1d8e" candidate_uuid: "13f0c3e0f54b458484b5a99d9b58fc82" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2b150c9788e84505a9f96579e7b19e75"
I20260809 15:36:11.684883 11882 raft_consensus.cc:3037] T 9c10a3fa1a7243e4afc60112863d1d8e P 2b150c9788e84505a9f96579e7b19e75 [term 0 FOLLOWER]: Advancing to term 1
I20260809 15:36:11.684890 11729 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9c10a3fa1a7243e4afc60112863d1d8e" candidate_uuid: "13f0c3e0f54b458484b5a99d9b58fc82" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d50738272bb8411c92663d6b4be495d0"
I20260809 15:36:11.685035 11729 raft_consensus.cc:3037] T 9c10a3fa1a7243e4afc60112863d1d8e P d50738272bb8411c92663d6b4be495d0 [term 0 FOLLOWER]: Advancing to term 1
I20260809 15:36:11.685827 11882 raft_consensus.cc:2454] T 9c10a3fa1a7243e4afc60112863d1d8e P 2b150c9788e84505a9f96579e7b19e75 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 13f0c3e0f54b458484b5a99d9b58fc82 in term 1.
I20260809 15:36:11.686115 11957 leader_election.cc:302] T 9c10a3fa1a7243e4afc60112863d1d8e P 13f0c3e0f54b458484b5a99d9b58fc82 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 13f0c3e0f54b458484b5a99d9b58fc82, 2b150c9788e84505a9f96579e7b19e75; no voters:
I20260809 15:36:11.686152 11729 raft_consensus.cc:2454] T 9c10a3fa1a7243e4afc60112863d1d8e P d50738272bb8411c92663d6b4be495d0 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 13f0c3e0f54b458484b5a99d9b58fc82 in term 1.
I20260809 15:36:11.686295 12122 raft_consensus.cc:2781] T 9c10a3fa1a7243e4afc60112863d1d8e P 13f0c3e0f54b458484b5a99d9b58fc82 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 15:36:11.686553 12122 raft_consensus.cc:686] T 9c10a3fa1a7243e4afc60112863d1d8e P 13f0c3e0f54b458484b5a99d9b58fc82 [term 1 LEADER]: Becoming Leader. State: Replica: 13f0c3e0f54b458484b5a99d9b58fc82, State: Running, Role: LEADER
I20260809 15:36:11.686880 12122 consensus_queue.cc:237] T 9c10a3fa1a7243e4afc60112863d1d8e P 13f0c3e0f54b458484b5a99d9b58fc82 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2b150c9788e84505a9f96579e7b19e75" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 34141 } } peers { permanent_uuid: "d50738272bb8411c92663d6b4be495d0" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 42041 } } peers { permanent_uuid: "13f0c3e0f54b458484b5a99d9b58fc82" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 43403 } }
I20260809 15:36:11.689816 11559 catalog_manager.cc:5495] T 9c10a3fa1a7243e4afc60112863d1d8e P 13f0c3e0f54b458484b5a99d9b58fc82 reported cstate change: term changed from 0 to 1, leader changed from <none> to 13f0c3e0f54b458484b5a99d9b58fc82 (127.3.89.195). New cstate: current_term: 1 leader_uuid: "13f0c3e0f54b458484b5a99d9b58fc82" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2b150c9788e84505a9f96579e7b19e75" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 34141 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d50738272bb8411c92663d6b4be495d0" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 42041 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "13f0c3e0f54b458484b5a99d9b58fc82" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 43403 } health_report { overall_health: HEALTHY } } }
I20260809 15:36:11.704838 12113 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:36:11.701190 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45146 (local address 127.3.89.195:43403)
0809 15:36:11.701340 (+ 150us) server_negotiation.cc:206] Beginning negotiation
0809 15:36:11.701345 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 15:36:11.701359 (+ 14us) server_negotiation.cc:407] Connection header received
0809 15:36:11.701553 (+ 194us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 15:36:11.701556 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 15:36:11.701607 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 15:36:11.701712 (+ 105us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 15:36:11.701977 (+ 265us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:36:11.702464 (+ 487us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:36:11.703435 (+ 971us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:36:11.703662 (+ 227us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:36:11.703772 (+ 110us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:36:11.704074 (+ 302us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 15:36:11.704458 (+ 384us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 15:36:11.704583 (+ 125us) server_negotiation.cc:1035] Waiting for connection context
0809 15:36:11.704646 (+ 63us) server_negotiation.cc:299] Negotiation successful
0809 15:36:11.704700 (+ 54us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":68}
I20260809 15:36:11.710152 11882 raft_consensus.cc:1261] T 9c10a3fa1a7243e4afc60112863d1d8e P 2b150c9788e84505a9f96579e7b19e75 [term 1 FOLLOWER]: Refusing update from remote peer 13f0c3e0f54b458484b5a99d9b58fc82: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 15:36:11.710152 11729 raft_consensus.cc:1261] T 9c10a3fa1a7243e4afc60112863d1d8e P d50738272bb8411c92663d6b4be495d0 [term 1 FOLLOWER]: Refusing update from remote peer 13f0c3e0f54b458484b5a99d9b58fc82: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 15:36:11.710853 12122 consensus_queue.cc:1035] T 9c10a3fa1a7243e4afc60112863d1d8e P 13f0c3e0f54b458484b5a99d9b58fc82 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d50738272bb8411c92663d6b4be495d0" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 42041 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 15:36:11.711055 12127 consensus_queue.cc:1035] T 9c10a3fa1a7243e4afc60112863d1d8e P 13f0c3e0f54b458484b5a99d9b58fc82 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2b150c9788e84505a9f96579e7b19e75" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 34141 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 15:36:11.718122 12131 mvcc.cc:205] Tried to move back new op lower bound from 7316642904918265856 to 7316642904832094208. Current Snapshot: MvccSnapshot[applied={T|T < 7316642904918265856}]
I20260809 15:36:11.719931 12133 mvcc.cc:205] Tried to move back new op lower bound from 7316642904918265856 to 7316642904832094208. Current Snapshot: MvccSnapshot[applied={T|T < 7316642904918265856}]
I20260809 15:36:11.727388 11559 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:39660:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 15:36:11.727547 11559 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:39660:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 15:36:11.729846 11559 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 6a9afd191e4f40c2a679860b4257c4a7: Sending DeleteTablet for 3 replicas of tablet 9c10a3fa1a7243e4afc60112863d1d8e
I20260809 15:36:11.730590 11862 tablet_service.cc:1514] Processing DeleteTablet for tablet 9c10a3fa1a7243e4afc60112863d1d8e with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 15:36:11 UTC) from {username='oryx', principal='oryx/127.3.89.254@KRBTEST.COM'} at 127.0.0.1:34258
I20260809 15:36:11.730749 12016 tablet_service.cc:1514] Processing DeleteTablet for tablet 9c10a3fa1a7243e4afc60112863d1d8e with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 15:36:11 UTC) from {username='oryx', principal='oryx/127.3.89.254@KRBTEST.COM'} at 127.0.0.1:45132
I20260809 15:36:11.731118 12143 tablet_replica.cc:330] T 9c10a3fa1a7243e4afc60112863d1d8e P 2b150c9788e84505a9f96579e7b19e75: stopping tablet replica
I20260809 15:36:11.731156 11707 tablet_service.cc:1514] Processing DeleteTablet for tablet 9c10a3fa1a7243e4afc60112863d1d8e with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 15:36:11 UTC) from {username='oryx', principal='oryx/127.3.89.254@KRBTEST.COM'} at 127.0.0.1:43584
I20260809 15:36:11.731236 12144 tablet_replica.cc:330] T 9c10a3fa1a7243e4afc60112863d1d8e P 13f0c3e0f54b458484b5a99d9b58fc82: stopping tablet replica
I20260809 15:36:11.731518 12144 raft_consensus.cc:2229] T 9c10a3fa1a7243e4afc60112863d1d8e P 13f0c3e0f54b458484b5a99d9b58fc82 [term 1 LEADER]: Raft consensus shutting down.
I20260809 15:36:11.731537 12143 raft_consensus.cc:2229] T 9c10a3fa1a7243e4afc60112863d1d8e P 2b150c9788e84505a9f96579e7b19e75 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 15:36:11.731943 12143 raft_consensus.cc:2258] T 9c10a3fa1a7243e4afc60112863d1d8e P 2b150c9788e84505a9f96579e7b19e75 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 15:36:11.731969 12144 raft_consensus.cc:2258] T 9c10a3fa1a7243e4afc60112863d1d8e P 13f0c3e0f54b458484b5a99d9b58fc82 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 15:36:11.732265 11559 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:39660:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260809 15:36:11.732764 11559 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260809 15:36:11.733445 12144 ts_tablet_manager.cc:1905] T 9c10a3fa1a7243e4afc60112863d1d8e P 13f0c3e0f54b458484b5a99d9b58fc82: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 15:36:11.733616 12145 tablet_replica.cc:330] T 9c10a3fa1a7243e4afc60112863d1d8e P d50738272bb8411c92663d6b4be495d0: stopping tablet replica
I20260809 15:36:11.733875 12145 raft_consensus.cc:2229] T 9c10a3fa1a7243e4afc60112863d1d8e P d50738272bb8411c92663d6b4be495d0 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 15:36:11.734136 12145 raft_consensus.cc:2258] T 9c10a3fa1a7243e4afc60112863d1d8e P d50738272bb8411c92663d6b4be495d0 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 15:36:11.735549 12145 ts_tablet_manager.cc:1905] T 9c10a3fa1a7243e4afc60112863d1d8e P d50738272bb8411c92663d6b4be495d0: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 15:36:11.736426 12144 ts_tablet_manager.cc:1918] T 9c10a3fa1a7243e4afc60112863d1d8e P 13f0c3e0f54b458484b5a99d9b58fc82: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 15:36:11.736517 12144 log.cc:1196] T 9c10a3fa1a7243e4afc60112863d1d8e P 13f0c3e0f54b458484b5a99d9b58fc82: Deleting WAL directory at /tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786289732823074-3431-0/minicluster-data/ts-2/wal/wals/9c10a3fa1a7243e4afc60112863d1d8e
I20260809 15:36:11.736838 12144 ts_tablet_manager.cc:1939] T 9c10a3fa1a7243e4afc60112863d1d8e P 13f0c3e0f54b458484b5a99d9b58fc82: Deleting consensus metadata
I20260809 15:36:11.737905 11545 catalog_manager.cc:4841] TS 13f0c3e0f54b458484b5a99d9b58fc82 (127.3.89.195:43403): tablet 9c10a3fa1a7243e4afc60112863d1d8e (table test-table [id=bde567cc8ba8435d86a7f918ed91346b]) successfully deleted
I20260809 15:36:11.738344 12145 ts_tablet_manager.cc:1918] T 9c10a3fa1a7243e4afc60112863d1d8e P d50738272bb8411c92663d6b4be495d0: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 15:36:11.738474 12145 log.cc:1196] T 9c10a3fa1a7243e4afc60112863d1d8e P d50738272bb8411c92663d6b4be495d0: Deleting WAL directory at /tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786289732823074-3431-0/minicluster-data/ts-0/wal/wals/9c10a3fa1a7243e4afc60112863d1d8e
I20260809 15:36:11.738827 12145 ts_tablet_manager.cc:1939] T 9c10a3fa1a7243e4afc60112863d1d8e P d50738272bb8411c92663d6b4be495d0: Deleting consensus metadata
I20260809 15:36:11.739709 11707 tablet_service.cc:1467] Processing CreateTablet for tablet 2b9ea43e2641439ab1cb03df8bfdf75d (DEFAULT_TABLE table=test-table [id=16739412250b46ecbac23ca39d820490]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 15:36:11.739969 11707 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 2b9ea43e2641439ab1cb03df8bfdf75d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 15:36:11.739952 11862 tablet_service.cc:1467] Processing CreateTablet for tablet 2b9ea43e2641439ab1cb03df8bfdf75d (DEFAULT_TABLE table=test-table [id=16739412250b46ecbac23ca39d820490]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 15:36:11.740195 11862 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 2b9ea43e2641439ab1cb03df8bfdf75d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 15:36:11.741724 12117 tablet_bootstrap.cc:492] T 2b9ea43e2641439ab1cb03df8bfdf75d P d50738272bb8411c92663d6b4be495d0: Bootstrap starting.
I20260809 15:36:11.742166 11546 catalog_manager.cc:4841] TS d50738272bb8411c92663d6b4be495d0 (127.3.89.193:42041): tablet 9c10a3fa1a7243e4afc60112863d1d8e (table test-table [id=bde567cc8ba8435d86a7f918ed91346b]) successfully deleted
I20260809 15:36:11.742682 12117 tablet_bootstrap.cc:654] T 2b9ea43e2641439ab1cb03df8bfdf75d P d50738272bb8411c92663d6b4be495d0: Neither blocks nor log segments found. Creating new log.
I20260809 15:36:11.743115 12115 tablet_bootstrap.cc:492] T 2b9ea43e2641439ab1cb03df8bfdf75d P 2b150c9788e84505a9f96579e7b19e75: Bootstrap starting.
I20260809 15:36:11.743101 12016 tablet_service.cc:1467] Processing CreateTablet for tablet 2b9ea43e2641439ab1cb03df8bfdf75d (DEFAULT_TABLE table=test-table [id=16739412250b46ecbac23ca39d820490]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 15:36:11.743343 12016 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 2b9ea43e2641439ab1cb03df8bfdf75d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 15:36:11.744035 12115 tablet_bootstrap.cc:654] T 2b9ea43e2641439ab1cb03df8bfdf75d P 2b150c9788e84505a9f96579e7b19e75: Neither blocks nor log segments found. Creating new log.
I20260809 15:36:11.745209 12115 tablet_bootstrap.cc:492] T 2b9ea43e2641439ab1cb03df8bfdf75d P 2b150c9788e84505a9f96579e7b19e75: No bootstrap required, opened a new log
I20260809 15:36:11.745232 12116 tablet_bootstrap.cc:492] T 2b9ea43e2641439ab1cb03df8bfdf75d P 13f0c3e0f54b458484b5a99d9b58fc82: Bootstrap starting.
I20260809 15:36:11.745287 12115 ts_tablet_manager.cc:1397] T 2b9ea43e2641439ab1cb03df8bfdf75d P 2b150c9788e84505a9f96579e7b19e75: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260809 15:36:11.745702 12115 raft_consensus.cc:348] T 2b9ea43e2641439ab1cb03df8bfdf75d P 2b150c9788e84505a9f96579e7b19e75 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d50738272bb8411c92663d6b4be495d0" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 42041 } } peers { permanent_uuid: "2b150c9788e84505a9f96579e7b19e75" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 34141 } } peers { permanent_uuid: "13f0c3e0f54b458484b5a99d9b58fc82" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 43403 } }
I20260809 15:36:11.745844 12115 raft_consensus.cc:374] T 2b9ea43e2641439ab1cb03df8bfdf75d P 2b150c9788e84505a9f96579e7b19e75 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 15:36:11.745874 12115 raft_consensus.cc:729] T 2b9ea43e2641439ab1cb03df8bfdf75d P 2b150c9788e84505a9f96579e7b19e75 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2b150c9788e84505a9f96579e7b19e75, State: Initialized, Role: FOLLOWER
I20260809 15:36:11.745996 12115 consensus_queue.cc:260] T 2b9ea43e2641439ab1cb03df8bfdf75d P 2b150c9788e84505a9f96579e7b19e75 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d50738272bb8411c92663d6b4be495d0" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 42041 } } peers { permanent_uuid: "2b150c9788e84505a9f96579e7b19e75" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 34141 } } peers { permanent_uuid: "13f0c3e0f54b458484b5a99d9b58fc82" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 43403 } }
I20260809 15:36:11.746189 12116 tablet_bootstrap.cc:654] T 2b9ea43e2641439ab1cb03df8bfdf75d P 13f0c3e0f54b458484b5a99d9b58fc82: Neither blocks nor log segments found. Creating new log.
I20260809 15:36:11.746263 12115 ts_tablet_manager.cc:1428] T 2b9ea43e2641439ab1cb03df8bfdf75d P 2b150c9788e84505a9f96579e7b19e75: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260809 15:36:11.747546 12143 ts_tablet_manager.cc:1905] T 9c10a3fa1a7243e4afc60112863d1d8e P 2b150c9788e84505a9f96579e7b19e75: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 15:36:11.747995 12117 tablet_bootstrap.cc:492] T 2b9ea43e2641439ab1cb03df8bfdf75d P d50738272bb8411c92663d6b4be495d0: No bootstrap required, opened a new log
I20260809 15:36:11.748067 12117 ts_tablet_manager.cc:1397] T 2b9ea43e2641439ab1cb03df8bfdf75d P d50738272bb8411c92663d6b4be495d0: Time spent bootstrapping tablet: real 0.006s user 0.002s sys 0.000s
I20260809 15:36:11.748485 12117 raft_consensus.cc:348] T 2b9ea43e2641439ab1cb03df8bfdf75d P d50738272bb8411c92663d6b4be495d0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d50738272bb8411c92663d6b4be495d0" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 42041 } } peers { permanent_uuid: "2b150c9788e84505a9f96579e7b19e75" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 34141 } } peers { permanent_uuid: "13f0c3e0f54b458484b5a99d9b58fc82" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 43403 } }
I20260809 15:36:11.748610 12117 raft_consensus.cc:374] T 2b9ea43e2641439ab1cb03df8bfdf75d P d50738272bb8411c92663d6b4be495d0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 15:36:11.748641 12117 raft_consensus.cc:729] T 2b9ea43e2641439ab1cb03df8bfdf75d P d50738272bb8411c92663d6b4be495d0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d50738272bb8411c92663d6b4be495d0, State: Initialized, Role: FOLLOWER
I20260809 15:36:11.748755 12117 consensus_queue.cc:260] T 2b9ea43e2641439ab1cb03df8bfdf75d P d50738272bb8411c92663d6b4be495d0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d50738272bb8411c92663d6b4be495d0" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 42041 } } peers { permanent_uuid: "2b150c9788e84505a9f96579e7b19e75" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 34141 } } peers { permanent_uuid: "13f0c3e0f54b458484b5a99d9b58fc82" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 43403 } }
I20260809 15:36:11.749502 12117 ts_tablet_manager.cc:1428] T 2b9ea43e2641439ab1cb03df8bfdf75d P d50738272bb8411c92663d6b4be495d0: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260809 15:36:11.750109 12116 tablet_bootstrap.cc:492] T 2b9ea43e2641439ab1cb03df8bfdf75d P 13f0c3e0f54b458484b5a99d9b58fc82: No bootstrap required, opened a new log
I20260809 15:36:11.750200 12116 ts_tablet_manager.cc:1397] T 2b9ea43e2641439ab1cb03df8bfdf75d P 13f0c3e0f54b458484b5a99d9b58fc82: Time spent bootstrapping tablet: real 0.005s user 0.002s sys 0.000s
I20260809 15:36:11.750553 12143 ts_tablet_manager.cc:1918] T 9c10a3fa1a7243e4afc60112863d1d8e P 2b150c9788e84505a9f96579e7b19e75: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 15:36:11.750648 12143 log.cc:1196] T 9c10a3fa1a7243e4afc60112863d1d8e P 2b150c9788e84505a9f96579e7b19e75: Deleting WAL directory at /tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786289732823074-3431-0/minicluster-data/ts-1/wal/wals/9c10a3fa1a7243e4afc60112863d1d8e
I20260809 15:36:11.750801 12116 raft_consensus.cc:348] T 2b9ea43e2641439ab1cb03df8bfdf75d P 13f0c3e0f54b458484b5a99d9b58fc82 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d50738272bb8411c92663d6b4be495d0" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 42041 } } peers { permanent_uuid: "2b150c9788e84505a9f96579e7b19e75" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 34141 } } peers { permanent_uuid: "13f0c3e0f54b458484b5a99d9b58fc82" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 43403 } }
I20260809 15:36:11.750907 12116 raft_consensus.cc:374] T 2b9ea43e2641439ab1cb03df8bfdf75d P 13f0c3e0f54b458484b5a99d9b58fc82 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 15:36:11.750952 12116 raft_consensus.cc:729] T 2b9ea43e2641439ab1cb03df8bfdf75d P 13f0c3e0f54b458484b5a99d9b58fc82 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 13f0c3e0f54b458484b5a99d9b58fc82, State: Initialized, Role: FOLLOWER
I20260809 15:36:11.751001 12143 ts_tablet_manager.cc:1939] T 9c10a3fa1a7243e4afc60112863d1d8e P 2b150c9788e84505a9f96579e7b19e75: Deleting consensus metadata
I20260809 15:36:11.751123 12116 consensus_queue.cc:260] T 2b9ea43e2641439ab1cb03df8bfdf75d P 13f0c3e0f54b458484b5a99d9b58fc82 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d50738272bb8411c92663d6b4be495d0" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 42041 } } peers { permanent_uuid: "2b150c9788e84505a9f96579e7b19e75" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 34141 } } peers { permanent_uuid: "13f0c3e0f54b458484b5a99d9b58fc82" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 43403 } }
I20260809 15:36:11.751392 12116 ts_tablet_manager.cc:1428] T 2b9ea43e2641439ab1cb03df8bfdf75d P 13f0c3e0f54b458484b5a99d9b58fc82: Time spent starting tablet: real 0.001s user 0.002s sys 0.000s
I20260809 15:36:11.752012 12121 raft_consensus.cc:482] T 2b9ea43e2641439ab1cb03df8bfdf75d P d50738272bb8411c92663d6b4be495d0 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 15:36:11.752104 12121 raft_consensus.cc:504] T 2b9ea43e2641439ab1cb03df8bfdf75d P d50738272bb8411c92663d6b4be495d0 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d50738272bb8411c92663d6b4be495d0" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 42041 } } peers { permanent_uuid: "2b150c9788e84505a9f96579e7b19e75" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 34141 } } peers { permanent_uuid: "13f0c3e0f54b458484b5a99d9b58fc82" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 43403 } }
I20260809 15:36:11.752192 11544 catalog_manager.cc:4841] TS 2b150c9788e84505a9f96579e7b19e75 (127.3.89.194:34141): tablet 9c10a3fa1a7243e4afc60112863d1d8e (table test-table [id=bde567cc8ba8435d86a7f918ed91346b]) successfully deleted
I20260809 15:36:11.753033 12121 leader_election.cc:288] T 2b9ea43e2641439ab1cb03df8bfdf75d P d50738272bb8411c92663d6b4be495d0 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 2b150c9788e84505a9f96579e7b19e75 (127.3.89.194:34141), 13f0c3e0f54b458484b5a99d9b58fc82 (127.3.89.195:43403)
I20260809 15:36:11.758116 12111 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:36:11.753396 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.89.193:41665 (local address 127.3.89.194:34141)
0809 15:36:11.754187 (+ 791us) server_negotiation.cc:206] Beginning negotiation
0809 15:36:11.754192 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 15:36:11.754205 (+ 13us) server_negotiation.cc:407] Connection header received
0809 15:36:11.754273 (+ 68us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 15:36:11.754276 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 15:36:11.754316 (+ 40us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 15:36:11.754409 (+ 93us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 15:36:11.754816 (+ 407us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:36:11.755653 (+ 837us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:36:11.756908 (+ 1255us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:36:11.757510 (+ 602us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:36:11.757629 (+ 119us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:36:11.757679 (+ 50us) server_negotiation.cc:1035] Waiting for connection context
0809 15:36:11.757907 (+ 228us) server_negotiation.cc:299] Negotiation successful
0809 15:36:11.757990 (+ 83us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":712}
I20260809 15:36:11.758122 12151 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:36:11.753280 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.89.194:34141 (local address 127.3.89.193:41665)
0809 15:36:11.753926 (+ 646us) negotiation.cc:107] Waiting for socket to connect
0809 15:36:11.753943 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0809 15:36:11.754043 (+ 100us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 15:36:11.754422 (+ 379us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 15:36:11.754424 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 15:36:11.754440 (+ 16us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 15:36:11.754661 (+ 221us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:36:11.754670 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:36:11.755804 (+ 1134us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:36:11.755807 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:36:11.756772 (+ 965us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:36:11.756781 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:36:11.757641 (+ 860us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:36:11.757645 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:36:11.757769 (+ 124us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:36:11.757796 (+ 27us) client_negotiation.cc:769] Sending connection context
0809 15:36:11.757875 (+ 79us) client_negotiation.cc:240] Negotiation successful
0809 15:36:11.757947 (+ 72us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":573,"mutex_wait_us":64,"spinlock_wait_cycles":21376,"thread_start_us":67,"threads_started":1}
I20260809 15:36:11.758795 11882 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2b9ea43e2641439ab1cb03df8bfdf75d" candidate_uuid: "d50738272bb8411c92663d6b4be495d0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2b150c9788e84505a9f96579e7b19e75" is_pre_election: true
I20260809 15:36:11.758865 12113 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:36:11.753396 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.89.193:53845 (local address 127.3.89.195:43403)
0809 15:36:11.753512 (+ 116us) server_negotiation.cc:206] Beginning negotiation
0809 15:36:11.753515 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0809 15:36:11.754100 (+ 585us) server_negotiation.cc:407] Connection header received
0809 15:36:11.754262 (+ 162us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 15:36:11.754266 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 15:36:11.754334 (+ 68us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 15:36:11.754406 (+ 72us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 15:36:11.754904 (+ 498us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:36:11.755671 (+ 767us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:36:11.757356 (+ 1685us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:36:11.757922 (+ 566us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:36:11.758040 (+ 118us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:36:11.758084 (+ 44us) server_negotiation.cc:1035] Waiting for connection context
0809 15:36:11.758663 (+ 579us) server_negotiation.cc:299] Negotiation successful
0809 15:36:11.758735 (+ 72us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":37}
I20260809 15:36:11.758930 11882 raft_consensus.cc:2454] T 2b9ea43e2641439ab1cb03df8bfdf75d P 2b150c9788e84505a9f96579e7b19e75 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d50738272bb8411c92663d6b4be495d0 in term 0.
I20260809 15:36:11.759404 12150 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:36:11.753281 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.89.195:43403 (local address 127.3.89.193:53845)
0809 15:36:11.754021 (+ 740us) negotiation.cc:107] Waiting for socket to connect
0809 15:36:11.754032 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0809 15:36:11.754160 (+ 128us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 15:36:11.754439 (+ 279us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 15:36:11.754442 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 15:36:11.754461 (+ 19us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 15:36:11.754665 (+ 204us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:36:11.754673 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:36:11.755798 (+ 1125us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:36:11.755801 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:36:11.757174 (+ 1373us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:36:11.757186 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:36:11.758148 (+ 962us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:36:11.758154 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:36:11.758489 (+ 335us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:36:11.758513 (+ 24us) client_negotiation.cc:769] Sending connection context
0809 15:36:11.758627 (+ 114us) client_negotiation.cc:240] Negotiation successful
0809 15:36:11.759252 (+ 625us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":687,"mutex_wait_us":419,"thread_start_us":106,"threads_started":1}
I20260809 15:36:11.759441 11654 leader_election.cc:302] T 2b9ea43e2641439ab1cb03df8bfdf75d P d50738272bb8411c92663d6b4be495d0 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2b150c9788e84505a9f96579e7b19e75, d50738272bb8411c92663d6b4be495d0; no voters:
I20260809 15:36:11.759718 12121 raft_consensus.cc:2781] T 2b9ea43e2641439ab1cb03df8bfdf75d P d50738272bb8411c92663d6b4be495d0 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260809 15:36:11.759794 12121 raft_consensus.cc:482] T 2b9ea43e2641439ab1cb03df8bfdf75d P d50738272bb8411c92663d6b4be495d0 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 15:36:11.759836 12121 raft_consensus.cc:3037] T 2b9ea43e2641439ab1cb03df8bfdf75d P d50738272bb8411c92663d6b4be495d0 [term 0 FOLLOWER]: Advancing to term 1
I20260809 15:36:11.759931 12037 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2b9ea43e2641439ab1cb03df8bfdf75d" candidate_uuid: "d50738272bb8411c92663d6b4be495d0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "13f0c3e0f54b458484b5a99d9b58fc82" is_pre_election: true
I20260809 15:36:11.760167 12037 raft_consensus.cc:2454] T 2b9ea43e2641439ab1cb03df8bfdf75d P 13f0c3e0f54b458484b5a99d9b58fc82 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d50738272bb8411c92663d6b4be495d0 in term 0.
I20260809 15:36:11.760995 12121 raft_consensus.cc:504] T 2b9ea43e2641439ab1cb03df8bfdf75d P d50738272bb8411c92663d6b4be495d0 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d50738272bb8411c92663d6b4be495d0" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 42041 } } peers { permanent_uuid: "2b150c9788e84505a9f96579e7b19e75" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 34141 } } peers { permanent_uuid: "13f0c3e0f54b458484b5a99d9b58fc82" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 43403 } }
I20260809 15:36:11.761365 12121 leader_election.cc:288] T 2b9ea43e2641439ab1cb03df8bfdf75d P d50738272bb8411c92663d6b4be495d0 [CANDIDATE]: Term 1 election: Requested vote from peers 2b150c9788e84505a9f96579e7b19e75 (127.3.89.194:34141), 13f0c3e0f54b458484b5a99d9b58fc82 (127.3.89.195:43403)
I20260809 15:36:11.761627 11882 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2b9ea43e2641439ab1cb03df8bfdf75d" candidate_uuid: "d50738272bb8411c92663d6b4be495d0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2b150c9788e84505a9f96579e7b19e75"
I20260809 15:36:11.761696 12037 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2b9ea43e2641439ab1cb03df8bfdf75d" candidate_uuid: "d50738272bb8411c92663d6b4be495d0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "13f0c3e0f54b458484b5a99d9b58fc82"
I20260809 15:36:11.761740 11882 raft_consensus.cc:3037] T 2b9ea43e2641439ab1cb03df8bfdf75d P 2b150c9788e84505a9f96579e7b19e75 [term 0 FOLLOWER]: Advancing to term 1
I20260809 15:36:11.761776 12037 raft_consensus.cc:3037] T 2b9ea43e2641439ab1cb03df8bfdf75d P 13f0c3e0f54b458484b5a99d9b58fc82 [term 0 FOLLOWER]: Advancing to term 1
I20260809 15:36:11.762621 12037 raft_consensus.cc:2454] T 2b9ea43e2641439ab1cb03df8bfdf75d P 13f0c3e0f54b458484b5a99d9b58fc82 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d50738272bb8411c92663d6b4be495d0 in term 1.
I20260809 15:36:11.762794 11882 raft_consensus.cc:2454] T 2b9ea43e2641439ab1cb03df8bfdf75d P 2b150c9788e84505a9f96579e7b19e75 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d50738272bb8411c92663d6b4be495d0 in term 1.
I20260809 15:36:11.762892 11656 leader_election.cc:302] T 2b9ea43e2641439ab1cb03df8bfdf75d P d50738272bb8411c92663d6b4be495d0 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 13f0c3e0f54b458484b5a99d9b58fc82, d50738272bb8411c92663d6b4be495d0; no voters:
I20260809 15:36:11.763134 12121 raft_consensus.cc:2781] T 2b9ea43e2641439ab1cb03df8bfdf75d P d50738272bb8411c92663d6b4be495d0 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 15:36:11.763391 12121 raft_consensus.cc:686] T 2b9ea43e2641439ab1cb03df8bfdf75d P d50738272bb8411c92663d6b4be495d0 [term 1 LEADER]: Becoming Leader. State: Replica: d50738272bb8411c92663d6b4be495d0, State: Running, Role: LEADER
I20260809 15:36:11.763628 12121 consensus_queue.cc:237] T 2b9ea43e2641439ab1cb03df8bfdf75d P d50738272bb8411c92663d6b4be495d0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d50738272bb8411c92663d6b4be495d0" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 42041 } } peers { permanent_uuid: "2b150c9788e84505a9f96579e7b19e75" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 34141 } } peers { permanent_uuid: "13f0c3e0f54b458484b5a99d9b58fc82" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 43403 } }
I20260809 15:36:11.766836 11559 catalog_manager.cc:5495] T 2b9ea43e2641439ab1cb03df8bfdf75d P d50738272bb8411c92663d6b4be495d0 reported cstate change: term changed from 0 to 1, leader changed from <none> to d50738272bb8411c92663d6b4be495d0 (127.3.89.193). New cstate: current_term: 1 leader_uuid: "d50738272bb8411c92663d6b4be495d0" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d50738272bb8411c92663d6b4be495d0" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 42041 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "2b150c9788e84505a9f96579e7b19e75" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 34141 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "13f0c3e0f54b458484b5a99d9b58fc82" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 43403 } health_report { overall_health: UNKNOWN } } }
W20260809 15:36:11.793119 12088 tablet.cc:2369] T 2b9ea43e2641439ab1cb03df8bfdf75d P 13f0c3e0f54b458484b5a99d9b58fc82: Can't schedule compaction. Clean time has not been advanced past its initial value.
Aug 09 15:36:11 dist-test-slave-2w3w krb5kdc[11515](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786289770, etypes {rep=17 tkt=17 ses=17}, oryx/127.3.89.254@KRBTEST.COM for oryx/127.3.89.254@KRBTEST.COM
I20260809 15:36:11.802045 12159 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:36:11.792685 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.89.254:38093 (local address 127.0.0.1:39662)
0809 15:36:11.793019 (+ 334us) negotiation.cc:107] Waiting for socket to connect
0809 15:36:11.793034 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0809 15:36:11.793160 (+ 126us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 15:36:11.793549 (+ 389us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 15:36:11.793552 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 15:36:11.793954 (+ 402us) client_negotiation.cc:189] Negotiated authn=SASL
0809 15:36:11.794059 (+ 105us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:36:11.794066 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:36:11.795157 (+ 1091us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:36:11.795159 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:36:11.795706 (+ 547us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:36:11.795713 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:36:11.796273 (+ 560us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:36:11.796276 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:36:11.796337 (+ 61us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:36:11.796965 (+ 628us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 15:36:11.796977 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0809 15:36:11.798686 (+ 1709us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 15:36:11.800547 (+ 1861us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 15:36:11.800555 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 15:36:11.800560 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0809 15:36:11.800775 (+ 215us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 15:36:11.801057 (+ 282us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 15:36:11.801059 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 15:36:11.801060 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0809 15:36:11.801135 (+ 75us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 15:36:11.801531 (+ 396us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 15:36:11.801535 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 15:36:11.801631 (+ 96us) client_negotiation.cc:769] Sending connection context
0809 15:36:11.801720 (+ 89us) client_negotiation.cc:240] Negotiation successful
0809 15:36:11.801903 (+ 183us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":246,"thread_start_us":134,"threads_started":1}
I20260809 15:36:11.802165 11613 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:36:11.792756 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39662 (local address 127.3.89.254:38093)
0809 15:36:11.792921 (+ 165us) server_negotiation.cc:206] Beginning negotiation
0809 15:36:11.792927 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 15:36:11.793128 (+ 201us) server_negotiation.cc:407] Connection header received
0809 15:36:11.793363 (+ 235us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 15:36:11.793366 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 15:36:11.793417 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 15:36:11.793503 (+ 86us) server_negotiation.cc:226] Negotiated authn=SASL
0809 15:36:11.794177 (+ 674us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:36:11.795056 (+ 879us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:36:11.795821 (+ 765us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:36:11.796161 (+ 340us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:36:11.796266 (+ 105us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:36:11.798826 (+ 2560us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 15:36:11.798830 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 15:36:11.798833 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 15:36:11.798858 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0809 15:36:11.800429 (+ 1571us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 15:36:11.800871 (+ 442us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 15:36:11.800875 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 15:36:11.800877 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 15:36:11.800967 (+ 90us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 15:36:11.801222 (+ 255us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 15:36:11.801225 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 15:36:11.801227 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 15:36:11.801429 (+ 202us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 15:36:11.801529 (+ 100us) server_negotiation.cc:1035] Waiting for connection context
0809 15:36:11.801809 (+ 280us) server_negotiation.cc:299] Negotiation successful
0809 15:36:11.801976 (+ 167us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":43,"server-negotiator.queue_time_us":47}
I20260809 15:36:11.806095 11559 catalog_manager.cc:1908] Servicing CreateTable request from {username='oryx', principal='oryx/127.3.89.254@KRBTEST.COM'} at 127.0.0.1:39662:
name: "kudu_system.kudu_transactions"
schema {
columns {
name: "txn_id"
type: INT64
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "entry_type"
type: INT8
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "identifier"
type: STRING
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "metadata"
type: STRING
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
rows: "<redacted>""\006\001\000\000\000\000\000\000\000\000\007\001@B\017\000\000\000\000\000"
indirect_data: "<redacted>"""
}
partition_schema {
range_schema {
columns {
name: "txn_id"
}
}
}
table_type: TXN_STATUS_TABLE
W20260809 15:36:11.806907 11559 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table kudu_system.kudu_transactions in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260809 15:36:11.812080 11707 tablet_service.cc:1467] Processing CreateTablet for tablet 1cc6356fc4d2423abfbccd0788d1fd9c (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=493fe9b8183c452eb24bb6ad924279c1]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260809 15:36:11.812353 12016 tablet_service.cc:1467] Processing CreateTablet for tablet 1cc6356fc4d2423abfbccd0788d1fd9c (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=493fe9b8183c452eb24bb6ad924279c1]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260809 15:36:11.812476 11707 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 1cc6356fc4d2423abfbccd0788d1fd9c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 15:36:11.812713 12016 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 1cc6356fc4d2423abfbccd0788d1fd9c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 15:36:11.814926 12116 tablet_bootstrap.cc:492] T 1cc6356fc4d2423abfbccd0788d1fd9c P 13f0c3e0f54b458484b5a99d9b58fc82: Bootstrap starting.
I20260809 15:36:11.815660 11862 tablet_service.cc:1467] Processing CreateTablet for tablet 1cc6356fc4d2423abfbccd0788d1fd9c (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=493fe9b8183c452eb24bb6ad924279c1]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260809 15:36:11.816025 11862 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 1cc6356fc4d2423abfbccd0788d1fd9c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 15:36:11.816098 12116 tablet_bootstrap.cc:654] T 1cc6356fc4d2423abfbccd0788d1fd9c P 13f0c3e0f54b458484b5a99d9b58fc82: Neither blocks nor log segments found. Creating new log.
I20260809 15:36:11.817420 12116 tablet_bootstrap.cc:492] T 1cc6356fc4d2423abfbccd0788d1fd9c P 13f0c3e0f54b458484b5a99d9b58fc82: No bootstrap required, opened a new log
I20260809 15:36:11.817528 12116 ts_tablet_manager.cc:1397] T 1cc6356fc4d2423abfbccd0788d1fd9c P 13f0c3e0f54b458484b5a99d9b58fc82: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260809 15:36:11.817955 12116 raft_consensus.cc:348] T 1cc6356fc4d2423abfbccd0788d1fd9c P 13f0c3e0f54b458484b5a99d9b58fc82 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d50738272bb8411c92663d6b4be495d0" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 42041 } } peers { permanent_uuid: "2b150c9788e84505a9f96579e7b19e75" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 34141 } } peers { permanent_uuid: "13f0c3e0f54b458484b5a99d9b58fc82" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 43403 } }
I20260809 15:36:11.818076 12116 raft_consensus.cc:374] T 1cc6356fc4d2423abfbccd0788d1fd9c P 13f0c3e0f54b458484b5a99d9b58fc82 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 15:36:11.818116 12116 raft_consensus.cc:729] T 1cc6356fc4d2423abfbccd0788d1fd9c P 13f0c3e0f54b458484b5a99d9b58fc82 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 13f0c3e0f54b458484b5a99d9b58fc82, State: Initialized, Role: FOLLOWER
I20260809 15:36:11.818270 12116 consensus_queue.cc:260] T 1cc6356fc4d2423abfbccd0788d1fd9c P 13f0c3e0f54b458484b5a99d9b58fc82 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d50738272bb8411c92663d6b4be495d0" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 42041 } } peers { permanent_uuid: "2b150c9788e84505a9f96579e7b19e75" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 34141 } } peers { permanent_uuid: "13f0c3e0f54b458484b5a99d9b58fc82" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 43403 } }
I20260809 15:36:11.818536 12122 tablet_replica.cc:439] T 1cc6356fc4d2423abfbccd0788d1fd9c P 13f0c3e0f54b458484b5a99d9b58fc82: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d50738272bb8411c92663d6b4be495d0" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 42041 } } peers { permanent_uuid: "2b150c9788e84505a9f96579e7b19e75" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 34141 } } peers { permanent_uuid: "13f0c3e0f54b458484b5a99d9b58fc82" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 43403 } } }
I20260809 15:36:11.818652 12122 tablet_replica.cc:442] T 1cc6356fc4d2423abfbccd0788d1fd9c P 13f0c3e0f54b458484b5a99d9b58fc82: This TxnStatusTablet replica's current role is: FOLLOWER
I20260809 15:36:11.818833 12115 tablet_bootstrap.cc:492] T 1cc6356fc4d2423abfbccd0788d1fd9c P 2b150c9788e84505a9f96579e7b19e75: Bootstrap starting.
I20260809 15:36:11.818888 12116 ts_tablet_manager.cc:1428] T 1cc6356fc4d2423abfbccd0788d1fd9c P 13f0c3e0f54b458484b5a99d9b58fc82: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260809 15:36:11.819779 12115 tablet_bootstrap.cc:654] T 1cc6356fc4d2423abfbccd0788d1fd9c P 2b150c9788e84505a9f96579e7b19e75: Neither blocks nor log segments found. Creating new log.
I20260809 15:36:11.820550 12117 tablet_bootstrap.cc:492] T 1cc6356fc4d2423abfbccd0788d1fd9c P d50738272bb8411c92663d6b4be495d0: Bootstrap starting.
I20260809 15:36:11.821484 12117 tablet_bootstrap.cc:654] T 1cc6356fc4d2423abfbccd0788d1fd9c P d50738272bb8411c92663d6b4be495d0: Neither blocks nor log segments found. Creating new log.
I20260809 15:36:11.822423 12115 tablet_bootstrap.cc:492] T 1cc6356fc4d2423abfbccd0788d1fd9c P 2b150c9788e84505a9f96579e7b19e75: No bootstrap required, opened a new log
I20260809 15:36:11.822503 12115 ts_tablet_manager.cc:1397] T 1cc6356fc4d2423abfbccd0788d1fd9c P 2b150c9788e84505a9f96579e7b19e75: Time spent bootstrapping tablet: real 0.004s user 0.002s sys 0.000s
I20260809 15:36:11.822909 12115 raft_consensus.cc:348] T 1cc6356fc4d2423abfbccd0788d1fd9c P 2b150c9788e84505a9f96579e7b19e75 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d50738272bb8411c92663d6b4be495d0" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 42041 } } peers { permanent_uuid: "2b150c9788e84505a9f96579e7b19e75" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 34141 } } peers { permanent_uuid: "13f0c3e0f54b458484b5a99d9b58fc82" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 43403 } }
I20260809 15:36:11.823043 12115 raft_consensus.cc:374] T 1cc6356fc4d2423abfbccd0788d1fd9c P 2b150c9788e84505a9f96579e7b19e75 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 15:36:11.823082 12115 raft_consensus.cc:729] T 1cc6356fc4d2423abfbccd0788d1fd9c P 2b150c9788e84505a9f96579e7b19e75 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2b150c9788e84505a9f96579e7b19e75, State: Initialized, Role: FOLLOWER
I20260809 15:36:11.823199 12115 consensus_queue.cc:260] T 1cc6356fc4d2423abfbccd0788d1fd9c P 2b150c9788e84505a9f96579e7b19e75 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d50738272bb8411c92663d6b4be495d0" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 42041 } } peers { permanent_uuid: "2b150c9788e84505a9f96579e7b19e75" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 34141 } } peers { permanent_uuid: "13f0c3e0f54b458484b5a99d9b58fc82" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 43403 } }
I20260809 15:36:11.823447 12117 tablet_bootstrap.cc:492] T 1cc6356fc4d2423abfbccd0788d1fd9c P d50738272bb8411c92663d6b4be495d0: No bootstrap required, opened a new log
I20260809 15:36:11.823467 12115 ts_tablet_manager.cc:1428] T 1cc6356fc4d2423abfbccd0788d1fd9c P 2b150c9788e84505a9f96579e7b19e75: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260809 15:36:11.823557 12117 ts_tablet_manager.cc:1397] T 1cc6356fc4d2423abfbccd0788d1fd9c P d50738272bb8411c92663d6b4be495d0: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260809 15:36:11.823678 12123 tablet_replica.cc:439] T 1cc6356fc4d2423abfbccd0788d1fd9c P 2b150c9788e84505a9f96579e7b19e75: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d50738272bb8411c92663d6b4be495d0" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 42041 } } peers { permanent_uuid: "2b150c9788e84505a9f96579e7b19e75" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 34141 } } peers { permanent_uuid: "13f0c3e0f54b458484b5a99d9b58fc82" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 43403 } } }
I20260809 15:36:11.823829 12123 tablet_replica.cc:442] T 1cc6356fc4d2423abfbccd0788d1fd9c P 2b150c9788e84505a9f96579e7b19e75: This TxnStatusTablet replica's current role is: FOLLOWER
I20260809 15:36:11.824008 12117 raft_consensus.cc:348] T 1cc6356fc4d2423abfbccd0788d1fd9c P d50738272bb8411c92663d6b4be495d0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d50738272bb8411c92663d6b4be495d0" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 42041 } } peers { permanent_uuid: "2b150c9788e84505a9f96579e7b19e75" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 34141 } } peers { permanent_uuid: "13f0c3e0f54b458484b5a99d9b58fc82" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 43403 } }
I20260809 15:36:11.824128 12117 raft_consensus.cc:374] T 1cc6356fc4d2423abfbccd0788d1fd9c P d50738272bb8411c92663d6b4be495d0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 15:36:11.824162 12117 raft_consensus.cc:729] T 1cc6356fc4d2423abfbccd0788d1fd9c P d50738272bb8411c92663d6b4be495d0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d50738272bb8411c92663d6b4be495d0, State: Initialized, Role: FOLLOWER
I20260809 15:36:11.824290 12117 consensus_queue.cc:260] T 1cc6356fc4d2423abfbccd0788d1fd9c P d50738272bb8411c92663d6b4be495d0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d50738272bb8411c92663d6b4be495d0" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 42041 } } peers { permanent_uuid: "2b150c9788e84505a9f96579e7b19e75" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 34141 } } peers { permanent_uuid: "13f0c3e0f54b458484b5a99d9b58fc82" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 43403 } }
I20260809 15:36:11.824559 12121 tablet_replica.cc:439] T 1cc6356fc4d2423abfbccd0788d1fd9c P d50738272bb8411c92663d6b4be495d0: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d50738272bb8411c92663d6b4be495d0" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 42041 } } peers { permanent_uuid: "2b150c9788e84505a9f96579e7b19e75" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 34141 } } peers { permanent_uuid: "13f0c3e0f54b458484b5a99d9b58fc82" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 43403 } } }
I20260809 15:36:11.824620 12117 ts_tablet_manager.cc:1428] T 1cc6356fc4d2423abfbccd0788d1fd9c P d50738272bb8411c92663d6b4be495d0: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20260809 15:36:11.824681 12121 tablet_replica.cc:442] T 1cc6356fc4d2423abfbccd0788d1fd9c P d50738272bb8411c92663d6b4be495d0: This TxnStatusTablet replica's current role is: FOLLOWER
W20260809 15:36:11.828299 11932 tablet.cc:2369] T 1cc6356fc4d2423abfbccd0788d1fd9c P 2b150c9788e84505a9f96579e7b19e75: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260809 15:36:11.854645 12123 raft_consensus.cc:482] T 1cc6356fc4d2423abfbccd0788d1fd9c P 2b150c9788e84505a9f96579e7b19e75 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 15:36:11.854816 12123 raft_consensus.cc:504] T 1cc6356fc4d2423abfbccd0788d1fd9c P 2b150c9788e84505a9f96579e7b19e75 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d50738272bb8411c92663d6b4be495d0" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 42041 } } peers { permanent_uuid: "2b150c9788e84505a9f96579e7b19e75" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 34141 } } peers { permanent_uuid: "13f0c3e0f54b458484b5a99d9b58fc82" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 43403 } }
I20260809 15:36:11.856088 12123 leader_election.cc:288] T 1cc6356fc4d2423abfbccd0788d1fd9c P 2b150c9788e84505a9f96579e7b19e75 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d50738272bb8411c92663d6b4be495d0 (127.3.89.193:42041), 13f0c3e0f54b458484b5a99d9b58fc82 (127.3.89.195:43403)
I20260809 15:36:11.860859 12114 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:36:11.856390 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.89.194:47995 (local address 127.3.89.193:42041)
0809 15:36:11.856525 (+ 135us) server_negotiation.cc:206] Beginning negotiation
0809 15:36:11.856529 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 15:36:11.856824 (+ 295us) server_negotiation.cc:407] Connection header received
0809 15:36:11.857050 (+ 226us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 15:36:11.857053 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 15:36:11.857107 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 15:36:11.857159 (+ 52us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 15:36:11.857595 (+ 436us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:36:11.858327 (+ 732us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:36:11.859567 (+ 1240us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:36:11.860235 (+ 668us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:36:11.860372 (+ 137us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:36:11.860424 (+ 52us) server_negotiation.cc:1035] Waiting for connection context
0809 15:36:11.860638 (+ 214us) server_negotiation.cc:299] Negotiation successful
0809 15:36:11.860736 (+ 98us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":54}
I20260809 15:36:11.860874 12165 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:36:11.856283 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.89.193:42041 (local address 127.3.89.194:47995)
0809 15:36:11.856715 (+ 432us) negotiation.cc:107] Waiting for socket to connect
0809 15:36:11.856741 (+ 26us) client_negotiation.cc:174] Beginning negotiation
0809 15:36:11.856905 (+ 164us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 15:36:11.857308 (+ 403us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 15:36:11.857311 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 15:36:11.857329 (+ 18us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 15:36:11.857434 (+ 105us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:36:11.857441 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:36:11.858444 (+ 1003us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:36:11.858447 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:36:11.859430 (+ 983us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:36:11.859439 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:36:11.860374 (+ 935us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:36:11.860378 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:36:11.860485 (+ 107us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:36:11.860517 (+ 32us) client_negotiation.cc:769] Sending connection context
0809 15:36:11.860613 (+ 96us) client_negotiation.cc:240] Negotiation successful
0809 15:36:11.860690 (+ 77us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":364,"mutex_wait_us":49,"thread_start_us":60,"threads_started":1}
I20260809 15:36:11.860965 12113 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:36:11.856665 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.89.194:49351 (local address 127.3.89.195:43403)
0809 15:36:11.856781 (+ 116us) server_negotiation.cc:206] Beginning negotiation
0809 15:36:11.856785 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 15:36:11.856804 (+ 19us) server_negotiation.cc:407] Connection header received
0809 15:36:11.856856 (+ 52us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 15:36:11.856859 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 15:36:11.856911 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 15:36:11.856978 (+ 67us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 15:36:11.857254 (+ 276us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:36:11.858031 (+ 777us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:36:11.859708 (+ 1677us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:36:11.860268 (+ 560us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:36:11.860372 (+ 104us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:36:11.860423 (+ 51us) server_negotiation.cc:1035] Waiting for connection context
0809 15:36:11.860755 (+ 332us) server_negotiation.cc:299] Negotiation successful
0809 15:36:11.860827 (+ 72us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":49}
I20260809 15:36:11.861373 12164 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:36:11.856283 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.89.195:43403 (local address 127.3.89.194:49351)
0809 15:36:11.856646 (+ 363us) negotiation.cc:107] Waiting for socket to connect
0809 15:36:11.856661 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0809 15:36:11.856760 (+ 99us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 15:36:11.856994 (+ 234us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 15:36:11.856996 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 15:36:11.857013 (+ 17us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 15:36:11.857134 (+ 121us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:36:11.857140 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:36:11.858156 (+ 1016us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:36:11.858160 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:36:11.859566 (+ 1406us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:36:11.859577 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:36:11.860494 (+ 917us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:36:11.860498 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:36:11.860607 (+ 109us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:36:11.860634 (+ 27us) client_negotiation.cc:769] Sending connection context
0809 15:36:11.861109 (+ 475us) client_negotiation.cc:240] Negotiation successful
0809 15:36:11.861202 (+ 93us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":291,"mutex_wait_us":68,"thread_start_us":74,"threads_started":1}
I20260809 15:36:11.861650 11729 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1cc6356fc4d2423abfbccd0788d1fd9c" candidate_uuid: "2b150c9788e84505a9f96579e7b19e75" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d50738272bb8411c92663d6b4be495d0" is_pre_election: true
I20260809 15:36:11.861826 11729 raft_consensus.cc:2454] T 1cc6356fc4d2423abfbccd0788d1fd9c P d50738272bb8411c92663d6b4be495d0 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2b150c9788e84505a9f96579e7b19e75 in term 0.
I20260809 15:36:11.862213 11808 leader_election.cc:302] T 1cc6356fc4d2423abfbccd0788d1fd9c P 2b150c9788e84505a9f96579e7b19e75 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2b150c9788e84505a9f96579e7b19e75, d50738272bb8411c92663d6b4be495d0; no voters:
I20260809 15:36:11.862257 12037 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1cc6356fc4d2423abfbccd0788d1fd9c" candidate_uuid: "2b150c9788e84505a9f96579e7b19e75" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "13f0c3e0f54b458484b5a99d9b58fc82" is_pre_election: true
I20260809 15:36:11.862382 12037 raft_consensus.cc:2454] T 1cc6356fc4d2423abfbccd0788d1fd9c P 13f0c3e0f54b458484b5a99d9b58fc82 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2b150c9788e84505a9f96579e7b19e75 in term 0.
I20260809 15:36:11.862498 12123 raft_consensus.cc:2781] T 1cc6356fc4d2423abfbccd0788d1fd9c P 2b150c9788e84505a9f96579e7b19e75 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260809 15:36:11.862562 12123 raft_consensus.cc:482] T 1cc6356fc4d2423abfbccd0788d1fd9c P 2b150c9788e84505a9f96579e7b19e75 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 15:36:11.862615 12123 raft_consensus.cc:3037] T 1cc6356fc4d2423abfbccd0788d1fd9c P 2b150c9788e84505a9f96579e7b19e75 [term 0 FOLLOWER]: Advancing to term 1
I20260809 15:36:11.863700 12123 raft_consensus.cc:504] T 1cc6356fc4d2423abfbccd0788d1fd9c P 2b150c9788e84505a9f96579e7b19e75 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d50738272bb8411c92663d6b4be495d0" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 42041 } } peers { permanent_uuid: "2b150c9788e84505a9f96579e7b19e75" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 34141 } } peers { permanent_uuid: "13f0c3e0f54b458484b5a99d9b58fc82" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 43403 } }
W20260809 15:36:11.864073 11780 tablet.cc:2369] T 1cc6356fc4d2423abfbccd0788d1fd9c P d50738272bb8411c92663d6b4be495d0: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260809 15:36:11.864125 12123 leader_election.cc:288] T 1cc6356fc4d2423abfbccd0788d1fd9c P 2b150c9788e84505a9f96579e7b19e75 [CANDIDATE]: Term 1 election: Requested vote from peers d50738272bb8411c92663d6b4be495d0 (127.3.89.193:42041), 13f0c3e0f54b458484b5a99d9b58fc82 (127.3.89.195:43403)
I20260809 15:36:11.864439 11729 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1cc6356fc4d2423abfbccd0788d1fd9c" candidate_uuid: "2b150c9788e84505a9f96579e7b19e75" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d50738272bb8411c92663d6b4be495d0"
I20260809 15:36:11.864554 11729 raft_consensus.cc:3037] T 1cc6356fc4d2423abfbccd0788d1fd9c P d50738272bb8411c92663d6b4be495d0 [term 0 FOLLOWER]: Advancing to term 1
I20260809 15:36:11.864526 12037 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1cc6356fc4d2423abfbccd0788d1fd9c" candidate_uuid: "2b150c9788e84505a9f96579e7b19e75" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "13f0c3e0f54b458484b5a99d9b58fc82"
I20260809 15:36:11.864639 12037 raft_consensus.cc:3037] T 1cc6356fc4d2423abfbccd0788d1fd9c P 13f0c3e0f54b458484b5a99d9b58fc82 [term 0 FOLLOWER]: Advancing to term 1
I20260809 15:36:11.865736 11729 raft_consensus.cc:2454] T 1cc6356fc4d2423abfbccd0788d1fd9c P d50738272bb8411c92663d6b4be495d0 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2b150c9788e84505a9f96579e7b19e75 in term 1.
I20260809 15:36:11.865747 12037 raft_consensus.cc:2454] T 1cc6356fc4d2423abfbccd0788d1fd9c P 13f0c3e0f54b458484b5a99d9b58fc82 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2b150c9788e84505a9f96579e7b19e75 in term 1.
I20260809 15:36:11.866051 11807 leader_election.cc:302] T 1cc6356fc4d2423abfbccd0788d1fd9c P 2b150c9788e84505a9f96579e7b19e75 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 13f0c3e0f54b458484b5a99d9b58fc82, 2b150c9788e84505a9f96579e7b19e75; no voters:
I20260809 15:36:11.866278 12123 raft_consensus.cc:2781] T 1cc6356fc4d2423abfbccd0788d1fd9c P 2b150c9788e84505a9f96579e7b19e75 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 15:36:11.866508 12123 raft_consensus.cc:686] T 1cc6356fc4d2423abfbccd0788d1fd9c P 2b150c9788e84505a9f96579e7b19e75 [term 1 LEADER]: Becoming Leader. State: Replica: 2b150c9788e84505a9f96579e7b19e75, State: Running, Role: LEADER
I20260809 15:36:11.866747 12123 consensus_queue.cc:237] T 1cc6356fc4d2423abfbccd0788d1fd9c P 2b150c9788e84505a9f96579e7b19e75 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d50738272bb8411c92663d6b4be495d0" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 42041 } } peers { permanent_uuid: "2b150c9788e84505a9f96579e7b19e75" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 34141 } } peers { permanent_uuid: "13f0c3e0f54b458484b5a99d9b58fc82" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 43403 } }
I20260809 15:36:11.868024 12166 tablet_replica.cc:439] T 1cc6356fc4d2423abfbccd0788d1fd9c P 2b150c9788e84505a9f96579e7b19e75: TxnStatusTablet state changed. Reason: New leader 2b150c9788e84505a9f96579e7b19e75. Latest consensus state: current_term: 1 leader_uuid: "2b150c9788e84505a9f96579e7b19e75" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d50738272bb8411c92663d6b4be495d0" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 42041 } } peers { permanent_uuid: "2b150c9788e84505a9f96579e7b19e75" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 34141 } } peers { permanent_uuid: "13f0c3e0f54b458484b5a99d9b58fc82" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 43403 } } }
I20260809 15:36:11.868141 12166 tablet_replica.cc:442] T 1cc6356fc4d2423abfbccd0788d1fd9c P 2b150c9788e84505a9f96579e7b19e75: This TxnStatusTablet replica's current role is: LEADER
I20260809 15:36:11.868643 12168 txn_status_manager.cc:873] Waiting until node catch up with all replicated operations in previous term...
I20260809 15:36:11.868795 12168 txn_status_manager.cc:929] Loading transaction status metadata into memory...
I20260809 15:36:11.869336 11560 catalog_manager.cc:5495] T 1cc6356fc4d2423abfbccd0788d1fd9c P 2b150c9788e84505a9f96579e7b19e75 reported cstate change: term changed from 0 to 1, leader changed from <none> to 2b150c9788e84505a9f96579e7b19e75 (127.3.89.194). New cstate: current_term: 1 leader_uuid: "2b150c9788e84505a9f96579e7b19e75" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d50738272bb8411c92663d6b4be495d0" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 42041 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "2b150c9788e84505a9f96579e7b19e75" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 34141 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "13f0c3e0f54b458484b5a99d9b58fc82" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 43403 } health_report { overall_health: UNKNOWN } } }
I20260809 15:36:11.890017 12111 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:36:11.885390 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34270 (local address 127.3.89.194:34141)
0809 15:36:11.885545 (+ 155us) server_negotiation.cc:206] Beginning negotiation
0809 15:36:11.885549 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 15:36:11.885621 (+ 72us) server_negotiation.cc:407] Connection header received
0809 15:36:11.885831 (+ 210us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 15:36:11.885834 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 15:36:11.885895 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 15:36:11.885997 (+ 102us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 15:36:11.886275 (+ 278us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:36:11.887081 (+ 806us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:36:11.888315 (+ 1234us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:36:11.888652 (+ 337us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:36:11.888767 (+ 115us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:36:11.889091 (+ 324us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 15:36:11.889581 (+ 490us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 15:36:11.889692 (+ 111us) server_negotiation.cc:1035] Waiting for connection context
0809 15:36:11.889790 (+ 98us) server_negotiation.cc:299] Negotiation successful
0809 15:36:11.889849 (+ 59us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
I20260809 15:36:11.890026 12159 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:36:11.885126 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.89.194:34141 (local address 127.0.0.1:34270)
0809 15:36:11.885557 (+ 431us) negotiation.cc:107] Waiting for socket to connect
0809 15:36:11.885573 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0809 15:36:11.885686 (+ 113us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 15:36:11.886035 (+ 349us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 15:36:11.886039 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 15:36:11.886048 (+ 9us) client_negotiation.cc:189] Negotiated authn=TOKEN
0809 15:36:11.886162 (+ 114us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:36:11.886168 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:36:11.887205 (+ 1037us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:36:11.887208 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:36:11.888187 (+ 979us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:36:11.888197 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:36:11.888778 (+ 581us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:36:11.888783 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:36:11.888888 (+ 105us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:36:11.888940 (+ 52us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0809 15:36:11.889694 (+ 754us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0809 15:36:11.889708 (+ 14us) client_negotiation.cc:769] Sending connection context
0809 15:36:11.889780 (+ 72us) client_negotiation.cc:240] Negotiation successful
0809 15:36:11.889875 (+ 95us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":324}
I20260809 15:36:11.893127 12037 raft_consensus.cc:1261] T 1cc6356fc4d2423abfbccd0788d1fd9c P 13f0c3e0f54b458484b5a99d9b58fc82 [term 1 FOLLOWER]: Refusing update from remote peer 2b150c9788e84505a9f96579e7b19e75: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 15:36:11.893759 11729 raft_consensus.cc:1261] T 1cc6356fc4d2423abfbccd0788d1fd9c P d50738272bb8411c92663d6b4be495d0 [term 1 FOLLOWER]: Refusing update from remote peer 2b150c9788e84505a9f96579e7b19e75: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 15:36:11.894246 12123 consensus_queue.cc:1035] T 1cc6356fc4d2423abfbccd0788d1fd9c P 2b150c9788e84505a9f96579e7b19e75 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d50738272bb8411c92663d6b4be495d0" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 42041 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 15:36:11.894485 12166 consensus_queue.cc:1035] T 1cc6356fc4d2423abfbccd0788d1fd9c P 2b150c9788e84505a9f96579e7b19e75 [LEADER]: Connected to new peer: Peer: permanent_uuid: "13f0c3e0f54b458484b5a99d9b58fc82" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 43403 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 15:36:11.895987 12121 tablet_replica.cc:439] T 1cc6356fc4d2423abfbccd0788d1fd9c P d50738272bb8411c92663d6b4be495d0: TxnStatusTablet state changed. Reason: New leader 2b150c9788e84505a9f96579e7b19e75. Latest consensus state: current_term: 1 leader_uuid: "2b150c9788e84505a9f96579e7b19e75" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d50738272bb8411c92663d6b4be495d0" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 42041 } } peers { permanent_uuid: "2b150c9788e84505a9f96579e7b19e75" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 34141 } } peers { permanent_uuid: "13f0c3e0f54b458484b5a99d9b58fc82" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 43403 } } }
I20260809 15:36:11.896112 12121 tablet_replica.cc:442] T 1cc6356fc4d2423abfbccd0788d1fd9c P d50738272bb8411c92663d6b4be495d0: This TxnStatusTablet replica's current role is: FOLLOWER
I20260809 15:36:11.898427 12122 tablet_replica.cc:439] T 1cc6356fc4d2423abfbccd0788d1fd9c P 13f0c3e0f54b458484b5a99d9b58fc82: TxnStatusTablet state changed. Reason: New leader 2b150c9788e84505a9f96579e7b19e75. Latest consensus state: current_term: 1 leader_uuid: "2b150c9788e84505a9f96579e7b19e75" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d50738272bb8411c92663d6b4be495d0" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 42041 } } peers { permanent_uuid: "2b150c9788e84505a9f96579e7b19e75" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 34141 } } peers { permanent_uuid: "13f0c3e0f54b458484b5a99d9b58fc82" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 43403 } } }
I20260809 15:36:11.898540 12122 tablet_replica.cc:442] T 1cc6356fc4d2423abfbccd0788d1fd9c P 13f0c3e0f54b458484b5a99d9b58fc82: This TxnStatusTablet replica's current role is: FOLLOWER
I20260809 15:36:11.900825 12166 tablet_replica.cc:439] T 1cc6356fc4d2423abfbccd0788d1fd9c P 2b150c9788e84505a9f96579e7b19e75: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "2b150c9788e84505a9f96579e7b19e75" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d50738272bb8411c92663d6b4be495d0" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 42041 } } peers { permanent_uuid: "2b150c9788e84505a9f96579e7b19e75" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 34141 } } peers { permanent_uuid: "13f0c3e0f54b458484b5a99d9b58fc82" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 43403 } } }
I20260809 15:36:11.900941 12166 tablet_replica.cc:442] T 1cc6356fc4d2423abfbccd0788d1fd9c P 2b150c9788e84505a9f96579e7b19e75: This TxnStatusTablet replica's current role is: LEADER
I20260809 15:36:11.901033 12134 mvcc.cc:205] Tried to move back new op lower bound from 7316642905668747264 to 7316642905568944128. Current Snapshot: MvccSnapshot[applied={T|T < 7316642905668747264}]
I20260809 15:36:11.902370 12122 tablet_replica.cc:439] T 1cc6356fc4d2423abfbccd0788d1fd9c P 13f0c3e0f54b458484b5a99d9b58fc82: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "2b150c9788e84505a9f96579e7b19e75" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d50738272bb8411c92663d6b4be495d0" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 42041 } } peers { permanent_uuid: "2b150c9788e84505a9f96579e7b19e75" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 34141 } } peers { permanent_uuid: "13f0c3e0f54b458484b5a99d9b58fc82" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 43403 } } }
I20260809 15:36:11.902506 12122 tablet_replica.cc:442] T 1cc6356fc4d2423abfbccd0788d1fd9c P 13f0c3e0f54b458484b5a99d9b58fc82: This TxnStatusTablet replica's current role is: FOLLOWER
I20260809 15:36:11.903000 12166 tablet_replica.cc:439] T 1cc6356fc4d2423abfbccd0788d1fd9c P 2b150c9788e84505a9f96579e7b19e75: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "2b150c9788e84505a9f96579e7b19e75" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d50738272bb8411c92663d6b4be495d0" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 42041 } } peers { permanent_uuid: "2b150c9788e84505a9f96579e7b19e75" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 34141 } } peers { permanent_uuid: "13f0c3e0f54b458484b5a99d9b58fc82" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 43403 } } }
I20260809 15:36:11.903105 12166 tablet_replica.cc:442] T 1cc6356fc4d2423abfbccd0788d1fd9c P 2b150c9788e84505a9f96579e7b19e75: This TxnStatusTablet replica's current role is: LEADER
I20260809 15:36:11.908450 12121 tablet_replica.cc:439] T 1cc6356fc4d2423abfbccd0788d1fd9c P d50738272bb8411c92663d6b4be495d0: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "2b150c9788e84505a9f96579e7b19e75" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d50738272bb8411c92663d6b4be495d0" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 42041 } } peers { permanent_uuid: "2b150c9788e84505a9f96579e7b19e75" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 34141 } } peers { permanent_uuid: "13f0c3e0f54b458484b5a99d9b58fc82" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 43403 } } }
I20260809 15:36:11.908560 12121 tablet_replica.cc:442] T 1cc6356fc4d2423abfbccd0788d1fd9c P d50738272bb8411c92663d6b4be495d0: This TxnStatusTablet replica's current role is: FOLLOWER
I20260809 15:36:11.912791 12114 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:36:11.908338 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43586 (local address 127.3.89.193:42041)
0809 15:36:11.908460 (+ 122us) server_negotiation.cc:206] Beginning negotiation
0809 15:36:11.908464 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 15:36:11.908477 (+ 13us) server_negotiation.cc:407] Connection header received
0809 15:36:11.908549 (+ 72us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 15:36:11.908552 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 15:36:11.908611 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 15:36:11.908707 (+ 96us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 15:36:11.909057 (+ 350us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:36:11.909815 (+ 758us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:36:11.911106 (+ 1291us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:36:11.911488 (+ 382us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:36:11.911590 (+ 102us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:36:11.911906 (+ 316us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 15:36:11.912406 (+ 500us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 15:36:11.912515 (+ 109us) server_negotiation.cc:1035] Waiting for connection context
0809 15:36:11.912608 (+ 93us) server_negotiation.cc:299] Negotiation successful
0809 15:36:11.912664 (+ 56us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":45}
I20260809 15:36:11.924311 12176 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:36:11.919802 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.89.194:34141 (local address 127.3.89.193:36673)
0809 15:36:11.920188 (+ 386us) negotiation.cc:107] Waiting for socket to connect
0809 15:36:11.920200 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0809 15:36:11.920319 (+ 119us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 15:36:11.920542 (+ 223us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 15:36:11.920545 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 15:36:11.920552 (+ 7us) client_negotiation.cc:189] Negotiated authn=TOKEN
0809 15:36:11.920660 (+ 108us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:36:11.920671 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:36:11.921393 (+ 722us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:36:11.921399 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:36:11.922837 (+ 1438us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:36:11.922848 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:36:11.923439 (+ 591us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:36:11.923442 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:36:11.923546 (+ 104us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:36:11.923669 (+ 123us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0809 15:36:11.924010 (+ 341us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0809 15:36:11.924018 (+ 8us) client_negotiation.cc:769] Sending connection context
0809 15:36:11.924087 (+ 69us) client_negotiation.cc:240] Negotiation successful
0809 15:36:11.924163 (+ 76us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":289,"thread_start_us":103,"threads_started":1}
I20260809 15:36:11.924301 12111 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:36:11.919892 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.89.193:36673 (local address 127.3.89.194:34141)
0809 15:36:11.920027 (+ 135us) server_negotiation.cc:206] Beginning negotiation
0809 15:36:11.920033 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 15:36:11.920284 (+ 251us) server_negotiation.cc:407] Connection header received
0809 15:36:11.920403 (+ 119us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 15:36:11.920405 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 15:36:11.920455 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 15:36:11.920533 (+ 78us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 15:36:11.920774 (+ 241us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:36:11.921281 (+ 507us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:36:11.923064 (+ 1783us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:36:11.923307 (+ 243us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:36:11.923431 (+ 124us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:36:11.923853 (+ 422us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 15:36:11.923922 (+ 69us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 15:36:11.924006 (+ 84us) server_negotiation.cc:1035] Waiting for connection context
0809 15:36:11.924111 (+ 105us) server_negotiation.cc:299] Negotiation successful
0809 15:36:11.924157 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":48}
I20260809 15:36:11.931218 12037 raft_consensus.cc:1261] T 2b9ea43e2641439ab1cb03df8bfdf75d P 13f0c3e0f54b458484b5a99d9b58fc82 [term 1 FOLLOWER]: Refusing update from remote peer d50738272bb8411c92663d6b4be495d0: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 15:36:11.931277 11882 raft_consensus.cc:1261] T 2b9ea43e2641439ab1cb03df8bfdf75d P 2b150c9788e84505a9f96579e7b19e75 [term 1 FOLLOWER]: Refusing update from remote peer d50738272bb8411c92663d6b4be495d0: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 15:36:11.931730 12121 consensus_queue.cc:1035] T 2b9ea43e2641439ab1cb03df8bfdf75d P d50738272bb8411c92663d6b4be495d0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "13f0c3e0f54b458484b5a99d9b58fc82" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 43403 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 15:36:11.931984 12152 consensus_queue.cc:1035] T 2b9ea43e2641439ab1cb03df8bfdf75d P d50738272bb8411c92663d6b4be495d0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2b150c9788e84505a9f96579e7b19e75" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 34141 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 15:36:11.955228 12114 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:36:11.951138 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.89.194:33077 (local address 127.3.89.193:42041)
0809 15:36:11.951268 (+ 130us) server_negotiation.cc:206] Beginning negotiation
0809 15:36:11.951274 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 15:36:11.951503 (+ 229us) server_negotiation.cc:407] Connection header received
0809 15:36:11.951645 (+ 142us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 15:36:11.951647 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 15:36:11.951692 (+ 45us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 15:36:11.951807 (+ 115us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 15:36:11.952228 (+ 421us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:36:11.952986 (+ 758us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:36:11.954051 (+ 1065us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:36:11.954283 (+ 232us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:36:11.954401 (+ 118us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:36:11.954714 (+ 313us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 15:36:11.954813 (+ 99us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 15:36:11.954891 (+ 78us) server_negotiation.cc:1035] Waiting for connection context
0809 15:36:11.955043 (+ 152us) server_negotiation.cc:299] Negotiation successful
0809 15:36:11.955091 (+ 48us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":49}
I20260809 15:36:11.955255 12182 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:36:11.951073 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.89.193:42041 (local address 127.3.89.194:33077)
0809 15:36:11.951406 (+ 333us) negotiation.cc:107] Waiting for socket to connect
0809 15:36:11.951419 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0809 15:36:11.951548 (+ 129us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 15:36:11.951816 (+ 268us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 15:36:11.951819 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 15:36:11.951843 (+ 24us) client_negotiation.cc:189] Negotiated authn=TOKEN
0809 15:36:11.951970 (+ 127us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:36:11.951976 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:36:11.953096 (+ 1120us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:36:11.953098 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:36:11.953949 (+ 851us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:36:11.953959 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:36:11.954435 (+ 476us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:36:11.954439 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:36:11.954541 (+ 102us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:36:11.954582 (+ 41us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0809 15:36:11.954896 (+ 314us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0809 15:36:11.954924 (+ 28us) client_negotiation.cc:769] Sending connection context
0809 15:36:11.955029 (+ 105us) client_negotiation.cc:240] Negotiation successful
0809 15:36:11.955107 (+ 78us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":219,"thread_start_us":129,"threads_started":1}
I20260809 15:36:11.979153 11559 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:39660:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 15:36:11.979331 11559 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:39660:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 15:36:11.981658 11559 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 6a9afd191e4f40c2a679860b4257c4a7: Sending DeleteTablet for 3 replicas of tablet 2b9ea43e2641439ab1cb03df8bfdf75d
I20260809 15:36:11.982220 11707 tablet_service.cc:1514] Processing DeleteTablet for tablet 2b9ea43e2641439ab1cb03df8bfdf75d with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 15:36:11 UTC) from {username='oryx', principal='oryx/127.3.89.254@KRBTEST.COM'} at 127.0.0.1:43584
I20260809 15:36:11.982414 12145 tablet_replica.cc:330] T 2b9ea43e2641439ab1cb03df8bfdf75d P d50738272bb8411c92663d6b4be495d0: stopping tablet replica
I20260809 15:36:11.982445 12016 tablet_service.cc:1514] Processing DeleteTablet for tablet 2b9ea43e2641439ab1cb03df8bfdf75d with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 15:36:11 UTC) from {username='oryx', principal='oryx/127.3.89.254@KRBTEST.COM'} at 127.0.0.1:45132
I20260809 15:36:11.982525 12145 raft_consensus.cc:2229] T 2b9ea43e2641439ab1cb03df8bfdf75d P d50738272bb8411c92663d6b4be495d0 [term 1 LEADER]: Raft consensus shutting down.
I20260809 15:36:11.982613 12144 tablet_replica.cc:330] T 2b9ea43e2641439ab1cb03df8bfdf75d P 13f0c3e0f54b458484b5a99d9b58fc82: stopping tablet replica
I20260809 15:36:11.982730 12144 raft_consensus.cc:2229] T 2b9ea43e2641439ab1cb03df8bfdf75d P 13f0c3e0f54b458484b5a99d9b58fc82 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 15:36:11.982800 12145 raft_consensus.cc:2258] T 2b9ea43e2641439ab1cb03df8bfdf75d P d50738272bb8411c92663d6b4be495d0 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 15:36:11.982851 12144 raft_consensus.cc:2258] T 2b9ea43e2641439ab1cb03df8bfdf75d P 13f0c3e0f54b458484b5a99d9b58fc82 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 15:36:11.983382 11862 tablet_service.cc:1514] Processing DeleteTablet for tablet 2b9ea43e2641439ab1cb03df8bfdf75d with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 15:36:11 UTC) from {username='oryx', principal='oryx/127.3.89.254@KRBTEST.COM'} at 127.0.0.1:34258
I20260809 15:36:11.983569 12143 tablet_replica.cc:330] T 2b9ea43e2641439ab1cb03df8bfdf75d P 2b150c9788e84505a9f96579e7b19e75: stopping tablet replica
I20260809 15:36:11.983701 12143 raft_consensus.cc:2229] T 2b9ea43e2641439ab1cb03df8bfdf75d P 2b150c9788e84505a9f96579e7b19e75 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 15:36:11.983817 12143 raft_consensus.cc:2258] T 2b9ea43e2641439ab1cb03df8bfdf75d P 2b150c9788e84505a9f96579e7b19e75 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 15:36:11.984061 12145 ts_tablet_manager.cc:1905] T 2b9ea43e2641439ab1cb03df8bfdf75d P d50738272bb8411c92663d6b4be495d0: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 15:36:11.984112 12144 ts_tablet_manager.cc:1905] T 2b9ea43e2641439ab1cb03df8bfdf75d P 13f0c3e0f54b458484b5a99d9b58fc82: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 15:36:11.986915 12144 ts_tablet_manager.cc:1918] T 2b9ea43e2641439ab1cb03df8bfdf75d P 13f0c3e0f54b458484b5a99d9b58fc82: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260809 15:36:11.987023 12144 log.cc:1196] T 2b9ea43e2641439ab1cb03df8bfdf75d P 13f0c3e0f54b458484b5a99d9b58fc82: Deleting WAL directory at /tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786289732823074-3431-0/minicluster-data/ts-2/wal/wals/2b9ea43e2641439ab1cb03df8bfdf75d
I20260809 15:36:11.987268 12144 ts_tablet_manager.cc:1939] T 2b9ea43e2641439ab1cb03df8bfdf75d P 13f0c3e0f54b458484b5a99d9b58fc82: Deleting consensus metadata
I20260809 15:36:11.987569 12145 ts_tablet_manager.cc:1918] T 2b9ea43e2641439ab1cb03df8bfdf75d P d50738272bb8411c92663d6b4be495d0: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260809 15:36:11.987653 12145 log.cc:1196] T 2b9ea43e2641439ab1cb03df8bfdf75d P d50738272bb8411c92663d6b4be495d0: Deleting WAL directory at /tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786289732823074-3431-0/minicluster-data/ts-0/wal/wals/2b9ea43e2641439ab1cb03df8bfdf75d
I20260809 15:36:11.987931 12145 ts_tablet_manager.cc:1939] T 2b9ea43e2641439ab1cb03df8bfdf75d P d50738272bb8411c92663d6b4be495d0: Deleting consensus metadata
I20260809 15:36:11.987979 11545 catalog_manager.cc:4841] TS 13f0c3e0f54b458484b5a99d9b58fc82 (127.3.89.195:43403): tablet 2b9ea43e2641439ab1cb03df8bfdf75d (table test-table [id=16739412250b46ecbac23ca39d820490]) successfully deleted
I20260809 15:36:11.988646 11546 catalog_manager.cc:4841] TS d50738272bb8411c92663d6b4be495d0 (127.3.89.193:42041): tablet 2b9ea43e2641439ab1cb03df8bfdf75d (table test-table [id=16739412250b46ecbac23ca39d820490]) successfully deleted
I20260809 15:36:11.988783 12143 ts_tablet_manager.cc:1905] T 2b9ea43e2641439ab1cb03df8bfdf75d P 2b150c9788e84505a9f96579e7b19e75: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 15:36:11.991520 12143 ts_tablet_manager.cc:1918] T 2b9ea43e2641439ab1cb03df8bfdf75d P 2b150c9788e84505a9f96579e7b19e75: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260809 15:36:11.991606 12143 log.cc:1196] T 2b9ea43e2641439ab1cb03df8bfdf75d P 2b150c9788e84505a9f96579e7b19e75: Deleting WAL directory at /tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786289732823074-3431-0/minicluster-data/ts-1/wal/wals/2b9ea43e2641439ab1cb03df8bfdf75d
I20260809 15:36:11.991879 12143 ts_tablet_manager.cc:1939] T 2b9ea43e2641439ab1cb03df8bfdf75d P 2b150c9788e84505a9f96579e7b19e75: Deleting consensus metadata
I20260809 15:36:11.992540 11544 catalog_manager.cc:4841] TS 2b150c9788e84505a9f96579e7b19e75 (127.3.89.194:34141): tablet 2b9ea43e2641439ab1cb03df8bfdf75d (table test-table [id=16739412250b46ecbac23ca39d820490]) successfully deleted
Aug 09 15:36:12 dist-test-slave-2w3w krb5kdc[11515](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786289772, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Aug 09 15:36:12 dist-test-slave-2w3w krb5kdc[11515](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, test-admin@KRBTEST.COM for kudu/127.3.89.254@KRBTEST.COM, Server not found in Kerberos database
Aug 09 15:36:12 dist-test-slave-2w3w krb5kdc[11515](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, test-admin@KRBTEST.COM for kudu/127.3.89.254@KRBTEST.COM, Server not found in Kerberos database
W20260809 15:36:12.022145 11613 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 15:36:12.013396 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39670 (local address 127.3.89.254:38093)
0809 15:36:12.013529 (+ 133us) server_negotiation.cc:206] Beginning negotiation
0809 15:36:12.013534 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 15:36:12.013784 (+ 250us) server_negotiation.cc:407] Connection header received
0809 15:36:12.014013 (+ 229us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 15:36:12.014017 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 15:36:12.014080 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 15:36:12.014183 (+ 103us) server_negotiation.cc:226] Negotiated authn=SASL
0809 15:36:12.015068 (+ 885us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:36:12.015988 (+ 920us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:36:12.017811 (+ 1823us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:36:12.018149 (+ 338us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:36:12.018269 (+ 120us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:36:12.022007 (+ 3738us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:39670: BlockingRecv error: recv got EOF from 127.0.0.1:39670 (error 108)
Metrics: {"server-negotiator.queue_time_us":56}
Aug 09 15:36:12 dist-test-slave-2w3w krb5kdc[11515](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786289772, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.3.89.254@KRBTEST.COM
I20260809 15:36:12.037782 11613 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:36:12.027356 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39674 (local address 127.3.89.254:38093)
0809 15:36:12.027471 (+ 115us) server_negotiation.cc:206] Beginning negotiation
0809 15:36:12.027475 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 15:36:12.027645 (+ 170us) server_negotiation.cc:407] Connection header received
0809 15:36:12.027763 (+ 118us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 15:36:12.027766 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 15:36:12.027811 (+ 45us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 15:36:12.027894 (+ 83us) server_negotiation.cc:226] Negotiated authn=SASL
0809 15:36:12.028632 (+ 738us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:36:12.029407 (+ 775us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:36:12.030534 (+ 1127us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:36:12.030872 (+ 338us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:36:12.031016 (+ 144us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:36:12.033957 (+ 2941us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 15:36:12.033961 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 15:36:12.033964 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 15:36:12.033991 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0809 15:36:12.035775 (+ 1784us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 15:36:12.036250 (+ 475us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 15:36:12.036254 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 15:36:12.036256 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 15:36:12.036345 (+ 89us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 15:36:12.036655 (+ 310us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 15:36:12.036658 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 15:36:12.036660 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 15:36:12.036889 (+ 229us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 15:36:12.037024 (+ 135us) server_negotiation.cc:1035] Waiting for connection context
0809 15:36:12.037293 (+ 269us) server_negotiation.cc:299] Negotiation successful
0809 15:36:12.037608 (+ 315us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":48}
I20260809 15:36:12.041157 11559 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:39674:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260809 15:36:12.041615 11559 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260809 15:36:12.047120 11707 tablet_service.cc:1467] Processing CreateTablet for tablet 872a08ec0e5244cabd7ff74f03179648 (DEFAULT_TABLE table=test-table [id=d3443ca3a45a442b8480236b4959c786]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 15:36:12.047434 11707 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 872a08ec0e5244cabd7ff74f03179648. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 15:36:12.047484 11862 tablet_service.cc:1467] Processing CreateTablet for tablet 872a08ec0e5244cabd7ff74f03179648 (DEFAULT_TABLE table=test-table [id=d3443ca3a45a442b8480236b4959c786]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 15:36:12.047755 11862 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 872a08ec0e5244cabd7ff74f03179648. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 15:36:12.050176 12117 tablet_bootstrap.cc:492] T 872a08ec0e5244cabd7ff74f03179648 P d50738272bb8411c92663d6b4be495d0: Bootstrap starting.
I20260809 15:36:12.050899 12115 tablet_bootstrap.cc:492] T 872a08ec0e5244cabd7ff74f03179648 P 2b150c9788e84505a9f96579e7b19e75: Bootstrap starting.
I20260809 15:36:12.051051 12117 tablet_bootstrap.cc:654] T 872a08ec0e5244cabd7ff74f03179648 P d50738272bb8411c92663d6b4be495d0: Neither blocks nor log segments found. Creating new log.
I20260809 15:36:12.051668 12115 tablet_bootstrap.cc:654] T 872a08ec0e5244cabd7ff74f03179648 P 2b150c9788e84505a9f96579e7b19e75: Neither blocks nor log segments found. Creating new log.
I20260809 15:36:12.051739 12016 tablet_service.cc:1467] Processing CreateTablet for tablet 872a08ec0e5244cabd7ff74f03179648 (DEFAULT_TABLE table=test-table [id=d3443ca3a45a442b8480236b4959c786]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 15:36:12.051992 12016 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 872a08ec0e5244cabd7ff74f03179648. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 15:36:12.053400 12115 tablet_bootstrap.cc:492] T 872a08ec0e5244cabd7ff74f03179648 P 2b150c9788e84505a9f96579e7b19e75: No bootstrap required, opened a new log
I20260809 15:36:12.053488 12115 ts_tablet_manager.cc:1397] T 872a08ec0e5244cabd7ff74f03179648 P 2b150c9788e84505a9f96579e7b19e75: Time spent bootstrapping tablet: real 0.003s user 0.001s sys 0.000s
I20260809 15:36:12.053864 12115 raft_consensus.cc:348] T 872a08ec0e5244cabd7ff74f03179648 P 2b150c9788e84505a9f96579e7b19e75 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d50738272bb8411c92663d6b4be495d0" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 42041 } } peers { permanent_uuid: "2b150c9788e84505a9f96579e7b19e75" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 34141 } } peers { permanent_uuid: "13f0c3e0f54b458484b5a99d9b58fc82" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 43403 } }
I20260809 15:36:12.053973 12115 raft_consensus.cc:374] T 872a08ec0e5244cabd7ff74f03179648 P 2b150c9788e84505a9f96579e7b19e75 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 15:36:12.054004 12115 raft_consensus.cc:729] T 872a08ec0e5244cabd7ff74f03179648 P 2b150c9788e84505a9f96579e7b19e75 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2b150c9788e84505a9f96579e7b19e75, State: Initialized, Role: FOLLOWER
I20260809 15:36:12.054108 12115 consensus_queue.cc:260] T 872a08ec0e5244cabd7ff74f03179648 P 2b150c9788e84505a9f96579e7b19e75 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d50738272bb8411c92663d6b4be495d0" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 42041 } } peers { permanent_uuid: "2b150c9788e84505a9f96579e7b19e75" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 34141 } } peers { permanent_uuid: "13f0c3e0f54b458484b5a99d9b58fc82" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 43403 } }
I20260809 15:36:12.054312 12116 tablet_bootstrap.cc:492] T 872a08ec0e5244cabd7ff74f03179648 P 13f0c3e0f54b458484b5a99d9b58fc82: Bootstrap starting.
I20260809 15:36:12.054339 12115 ts_tablet_manager.cc:1428] T 872a08ec0e5244cabd7ff74f03179648 P 2b150c9788e84505a9f96579e7b19e75: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260809 15:36:12.055248 12116 tablet_bootstrap.cc:654] T 872a08ec0e5244cabd7ff74f03179648 P 13f0c3e0f54b458484b5a99d9b58fc82: Neither blocks nor log segments found. Creating new log.
I20260809 15:36:12.056486 12116 tablet_bootstrap.cc:492] T 872a08ec0e5244cabd7ff74f03179648 P 13f0c3e0f54b458484b5a99d9b58fc82: No bootstrap required, opened a new log
I20260809 15:36:12.056588 12116 ts_tablet_manager.cc:1397] T 872a08ec0e5244cabd7ff74f03179648 P 13f0c3e0f54b458484b5a99d9b58fc82: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260809 15:36:12.056931 12116 raft_consensus.cc:348] T 872a08ec0e5244cabd7ff74f03179648 P 13f0c3e0f54b458484b5a99d9b58fc82 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d50738272bb8411c92663d6b4be495d0" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 42041 } } peers { permanent_uuid: "2b150c9788e84505a9f96579e7b19e75" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 34141 } } peers { permanent_uuid: "13f0c3e0f54b458484b5a99d9b58fc82" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 43403 } }
I20260809 15:36:12.057041 12116 raft_consensus.cc:374] T 872a08ec0e5244cabd7ff74f03179648 P 13f0c3e0f54b458484b5a99d9b58fc82 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 15:36:12.057091 12116 raft_consensus.cc:729] T 872a08ec0e5244cabd7ff74f03179648 P 13f0c3e0f54b458484b5a99d9b58fc82 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 13f0c3e0f54b458484b5a99d9b58fc82, State: Initialized, Role: FOLLOWER
I20260809 15:36:12.057224 12116 consensus_queue.cc:260] T 872a08ec0e5244cabd7ff74f03179648 P 13f0c3e0f54b458484b5a99d9b58fc82 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d50738272bb8411c92663d6b4be495d0" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 42041 } } peers { permanent_uuid: "2b150c9788e84505a9f96579e7b19e75" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 34141 } } peers { permanent_uuid: "13f0c3e0f54b458484b5a99d9b58fc82" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 43403 } }
I20260809 15:36:12.057538 12116 ts_tablet_manager.cc:1428] T 872a08ec0e5244cabd7ff74f03179648 P 13f0c3e0f54b458484b5a99d9b58fc82: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260809 15:36:12.058542 12117 tablet_bootstrap.cc:492] T 872a08ec0e5244cabd7ff74f03179648 P d50738272bb8411c92663d6b4be495d0: No bootstrap required, opened a new log
I20260809 15:36:12.058637 12117 ts_tablet_manager.cc:1397] T 872a08ec0e5244cabd7ff74f03179648 P d50738272bb8411c92663d6b4be495d0: Time spent bootstrapping tablet: real 0.009s user 0.000s sys 0.001s
I20260809 15:36:12.059149 12117 raft_consensus.cc:348] T 872a08ec0e5244cabd7ff74f03179648 P d50738272bb8411c92663d6b4be495d0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d50738272bb8411c92663d6b4be495d0" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 42041 } } peers { permanent_uuid: "2b150c9788e84505a9f96579e7b19e75" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 34141 } } peers { permanent_uuid: "13f0c3e0f54b458484b5a99d9b58fc82" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 43403 } }
I20260809 15:36:12.059273 12117 raft_consensus.cc:374] T 872a08ec0e5244cabd7ff74f03179648 P d50738272bb8411c92663d6b4be495d0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 15:36:12.059315 12117 raft_consensus.cc:729] T 872a08ec0e5244cabd7ff74f03179648 P d50738272bb8411c92663d6b4be495d0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d50738272bb8411c92663d6b4be495d0, State: Initialized, Role: FOLLOWER
I20260809 15:36:12.059458 12117 consensus_queue.cc:260] T 872a08ec0e5244cabd7ff74f03179648 P d50738272bb8411c92663d6b4be495d0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d50738272bb8411c92663d6b4be495d0" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 42041 } } peers { permanent_uuid: "2b150c9788e84505a9f96579e7b19e75" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 34141 } } peers { permanent_uuid: "13f0c3e0f54b458484b5a99d9b58fc82" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 43403 } }
I20260809 15:36:12.059739 12117 ts_tablet_manager.cc:1428] T 872a08ec0e5244cabd7ff74f03179648 P d50738272bb8411c92663d6b4be495d0: Time spent starting tablet: real 0.001s user 0.000s sys 0.001s
I20260809 15:36:12.302680 12177 raft_consensus.cc:482] T 872a08ec0e5244cabd7ff74f03179648 P 2b150c9788e84505a9f96579e7b19e75 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 15:36:12.302845 12177 raft_consensus.cc:504] T 872a08ec0e5244cabd7ff74f03179648 P 2b150c9788e84505a9f96579e7b19e75 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d50738272bb8411c92663d6b4be495d0" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 42041 } } peers { permanent_uuid: "2b150c9788e84505a9f96579e7b19e75" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 34141 } } peers { permanent_uuid: "13f0c3e0f54b458484b5a99d9b58fc82" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 43403 } }
I20260809 15:36:12.303305 12177 leader_election.cc:288] T 872a08ec0e5244cabd7ff74f03179648 P 2b150c9788e84505a9f96579e7b19e75 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d50738272bb8411c92663d6b4be495d0 (127.3.89.193:42041), 13f0c3e0f54b458484b5a99d9b58fc82 (127.3.89.195:43403)
I20260809 15:36:12.303803 11729 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "872a08ec0e5244cabd7ff74f03179648" candidate_uuid: "2b150c9788e84505a9f96579e7b19e75" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d50738272bb8411c92663d6b4be495d0" is_pre_election: true
I20260809 15:36:12.303802 12037 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "872a08ec0e5244cabd7ff74f03179648" candidate_uuid: "2b150c9788e84505a9f96579e7b19e75" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "13f0c3e0f54b458484b5a99d9b58fc82" is_pre_election: true
I20260809 15:36:12.303943 11729 raft_consensus.cc:2454] T 872a08ec0e5244cabd7ff74f03179648 P d50738272bb8411c92663d6b4be495d0 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2b150c9788e84505a9f96579e7b19e75 in term 0.
I20260809 15:36:12.303993 12037 raft_consensus.cc:2454] T 872a08ec0e5244cabd7ff74f03179648 P 13f0c3e0f54b458484b5a99d9b58fc82 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2b150c9788e84505a9f96579e7b19e75 in term 0.
I20260809 15:36:12.304325 11808 leader_election.cc:302] T 872a08ec0e5244cabd7ff74f03179648 P 2b150c9788e84505a9f96579e7b19e75 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2b150c9788e84505a9f96579e7b19e75, d50738272bb8411c92663d6b4be495d0; no voters:
I20260809 15:36:12.304557 12177 raft_consensus.cc:2781] T 872a08ec0e5244cabd7ff74f03179648 P 2b150c9788e84505a9f96579e7b19e75 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260809 15:36:12.304629 12177 raft_consensus.cc:482] T 872a08ec0e5244cabd7ff74f03179648 P 2b150c9788e84505a9f96579e7b19e75 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 15:36:12.304692 12177 raft_consensus.cc:3037] T 872a08ec0e5244cabd7ff74f03179648 P 2b150c9788e84505a9f96579e7b19e75 [term 0 FOLLOWER]: Advancing to term 1
I20260809 15:36:12.305725 12177 raft_consensus.cc:504] T 872a08ec0e5244cabd7ff74f03179648 P 2b150c9788e84505a9f96579e7b19e75 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d50738272bb8411c92663d6b4be495d0" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 42041 } } peers { permanent_uuid: "2b150c9788e84505a9f96579e7b19e75" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 34141 } } peers { permanent_uuid: "13f0c3e0f54b458484b5a99d9b58fc82" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 43403 } }
I20260809 15:36:12.306097 12177 leader_election.cc:288] T 872a08ec0e5244cabd7ff74f03179648 P 2b150c9788e84505a9f96579e7b19e75 [CANDIDATE]: Term 1 election: Requested vote from peers d50738272bb8411c92663d6b4be495d0 (127.3.89.193:42041), 13f0c3e0f54b458484b5a99d9b58fc82 (127.3.89.195:43403)
I20260809 15:36:12.306460 12037 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "872a08ec0e5244cabd7ff74f03179648" candidate_uuid: "2b150c9788e84505a9f96579e7b19e75" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "13f0c3e0f54b458484b5a99d9b58fc82"
I20260809 15:36:12.306440 11729 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "872a08ec0e5244cabd7ff74f03179648" candidate_uuid: "2b150c9788e84505a9f96579e7b19e75" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d50738272bb8411c92663d6b4be495d0"
I20260809 15:36:12.306588 12037 raft_consensus.cc:3037] T 872a08ec0e5244cabd7ff74f03179648 P 13f0c3e0f54b458484b5a99d9b58fc82 [term 0 FOLLOWER]: Advancing to term 1
I20260809 15:36:12.306605 11729 raft_consensus.cc:3037] T 872a08ec0e5244cabd7ff74f03179648 P d50738272bb8411c92663d6b4be495d0 [term 0 FOLLOWER]: Advancing to term 1
I20260809 15:36:12.307624 12037 raft_consensus.cc:2454] T 872a08ec0e5244cabd7ff74f03179648 P 13f0c3e0f54b458484b5a99d9b58fc82 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2b150c9788e84505a9f96579e7b19e75 in term 1.
I20260809 15:36:12.307636 11729 raft_consensus.cc:2454] T 872a08ec0e5244cabd7ff74f03179648 P d50738272bb8411c92663d6b4be495d0 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2b150c9788e84505a9f96579e7b19e75 in term 1.
I20260809 15:36:12.307942 11807 leader_election.cc:302] T 872a08ec0e5244cabd7ff74f03179648 P 2b150c9788e84505a9f96579e7b19e75 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 13f0c3e0f54b458484b5a99d9b58fc82, 2b150c9788e84505a9f96579e7b19e75; no voters:
I20260809 15:36:12.308135 12177 raft_consensus.cc:2781] T 872a08ec0e5244cabd7ff74f03179648 P 2b150c9788e84505a9f96579e7b19e75 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 15:36:12.308229 12177 raft_consensus.cc:686] T 872a08ec0e5244cabd7ff74f03179648 P 2b150c9788e84505a9f96579e7b19e75 [term 1 LEADER]: Becoming Leader. State: Replica: 2b150c9788e84505a9f96579e7b19e75, State: Running, Role: LEADER
I20260809 15:36:12.308385 12177 consensus_queue.cc:237] T 872a08ec0e5244cabd7ff74f03179648 P 2b150c9788e84505a9f96579e7b19e75 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d50738272bb8411c92663d6b4be495d0" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 42041 } } peers { permanent_uuid: "2b150c9788e84505a9f96579e7b19e75" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 34141 } } peers { permanent_uuid: "13f0c3e0f54b458484b5a99d9b58fc82" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 43403 } }
I20260809 15:36:12.310000 11559 catalog_manager.cc:5495] T 872a08ec0e5244cabd7ff74f03179648 P 2b150c9788e84505a9f96579e7b19e75 reported cstate change: term changed from 0 to 1, leader changed from <none> to 2b150c9788e84505a9f96579e7b19e75 (127.3.89.194). New cstate: current_term: 1 leader_uuid: "2b150c9788e84505a9f96579e7b19e75" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d50738272bb8411c92663d6b4be495d0" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 42041 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "2b150c9788e84505a9f96579e7b19e75" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 34141 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "13f0c3e0f54b458484b5a99d9b58fc82" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 43403 } health_report { overall_health: UNKNOWN } } }
I20260809 15:36:12.350054 12111 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:36:12.347005 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34286 (local address 127.3.89.194:34141)
0809 15:36:12.347131 (+ 126us) server_negotiation.cc:206] Beginning negotiation
0809 15:36:12.347136 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 15:36:12.347177 (+ 41us) server_negotiation.cc:407] Connection header received
0809 15:36:12.347295 (+ 118us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 15:36:12.347298 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 15:36:12.347346 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 15:36:12.347423 (+ 77us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 15:36:12.347658 (+ 235us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:36:12.348177 (+ 519us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:36:12.349002 (+ 825us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:36:12.349207 (+ 205us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:36:12.349303 (+ 96us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:36:12.349627 (+ 324us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 15:36:12.349694 (+ 67us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 15:36:12.349757 (+ 63us) server_negotiation.cc:1035] Waiting for connection context
0809 15:36:12.349840 (+ 83us) server_negotiation.cc:299] Negotiation successful
0809 15:36:12.349882 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":40}
I20260809 15:36:12.352411 11729 raft_consensus.cc:1261] T 872a08ec0e5244cabd7ff74f03179648 P d50738272bb8411c92663d6b4be495d0 [term 1 FOLLOWER]: Refusing update from remote peer 2b150c9788e84505a9f96579e7b19e75: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 15:36:12.352411 12037 raft_consensus.cc:1261] T 872a08ec0e5244cabd7ff74f03179648 P 13f0c3e0f54b458484b5a99d9b58fc82 [term 1 FOLLOWER]: Refusing update from remote peer 2b150c9788e84505a9f96579e7b19e75: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 15:36:12.352890 12173 consensus_queue.cc:1035] T 872a08ec0e5244cabd7ff74f03179648 P 2b150c9788e84505a9f96579e7b19e75 [LEADER]: Connected to new peer: Peer: permanent_uuid: "13f0c3e0f54b458484b5a99d9b58fc82" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 43403 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 15:36:12.353096 12177 consensus_queue.cc:1035] T 872a08ec0e5244cabd7ff74f03179648 P 2b150c9788e84505a9f96579e7b19e75 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d50738272bb8411c92663d6b4be495d0" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 42041 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 15:36:12.362736 11560 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:39674:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 15:36:12.362900 11560 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:39674:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 15:36:12.365350 11560 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 6a9afd191e4f40c2a679860b4257c4a7: Sending DeleteTablet for 3 replicas of tablet 872a08ec0e5244cabd7ff74f03179648
I20260809 15:36:12.365924 12016 tablet_service.cc:1514] Processing DeleteTablet for tablet 872a08ec0e5244cabd7ff74f03179648 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 15:36:12 UTC) from {username='oryx', principal='oryx/127.3.89.254@KRBTEST.COM'} at 127.0.0.1:45132
I20260809 15:36:12.366001 11862 tablet_service.cc:1514] Processing DeleteTablet for tablet 872a08ec0e5244cabd7ff74f03179648 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 15:36:12 UTC) from {username='oryx', principal='oryx/127.3.89.254@KRBTEST.COM'} at 127.0.0.1:34258
I20260809 15:36:12.366104 12144 tablet_replica.cc:330] T 872a08ec0e5244cabd7ff74f03179648 P 13f0c3e0f54b458484b5a99d9b58fc82: stopping tablet replica
I20260809 15:36:12.366173 12143 tablet_replica.cc:330] T 872a08ec0e5244cabd7ff74f03179648 P 2b150c9788e84505a9f96579e7b19e75: stopping tablet replica
I20260809 15:36:12.366171 11707 tablet_service.cc:1514] Processing DeleteTablet for tablet 872a08ec0e5244cabd7ff74f03179648 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 15:36:12 UTC) from {username='oryx', principal='oryx/127.3.89.254@KRBTEST.COM'} at 127.0.0.1:43584
I20260809 15:36:12.366219 12144 raft_consensus.cc:2229] T 872a08ec0e5244cabd7ff74f03179648 P 13f0c3e0f54b458484b5a99d9b58fc82 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 15:36:12.366297 12143 raft_consensus.cc:2229] T 872a08ec0e5244cabd7ff74f03179648 P 2b150c9788e84505a9f96579e7b19e75 [term 1 LEADER]: Raft consensus shutting down.
I20260809 15:36:12.366328 11706 tablet_service.cc:1514] Processing DeleteTablet for tablet 872a08ec0e5244cabd7ff74f03179648 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 15:36:12 UTC) from {username='oryx', principal='oryx/127.3.89.254@KRBTEST.COM'} at 127.0.0.1:43584
I20260809 15:36:12.366357 12144 raft_consensus.cc:2258] T 872a08ec0e5244cabd7ff74f03179648 P 13f0c3e0f54b458484b5a99d9b58fc82 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 15:36:12.366583 12143 raft_consensus.cc:2258] T 872a08ec0e5244cabd7ff74f03179648 P 2b150c9788e84505a9f96579e7b19e75 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 15:36:12.366781 12145 tablet_replica.cc:330] T 872a08ec0e5244cabd7ff74f03179648 P d50738272bb8411c92663d6b4be495d0: stopping tablet replica
I20260809 15:36:12.366899 12145 raft_consensus.cc:2229] T 872a08ec0e5244cabd7ff74f03179648 P d50738272bb8411c92663d6b4be495d0 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 15:36:12.367069 12145 raft_consensus.cc:2258] T 872a08ec0e5244cabd7ff74f03179648 P d50738272bb8411c92663d6b4be495d0 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 15:36:12.367791 12143 ts_tablet_manager.cc:1905] T 872a08ec0e5244cabd7ff74f03179648 P 2b150c9788e84505a9f96579e7b19e75: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 15:36:12.369760 12143 ts_tablet_manager.cc:1918] T 872a08ec0e5244cabd7ff74f03179648 P 2b150c9788e84505a9f96579e7b19e75: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 15:36:12.369853 12143 log.cc:1196] T 872a08ec0e5244cabd7ff74f03179648 P 2b150c9788e84505a9f96579e7b19e75: Deleting WAL directory at /tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786289732823074-3431-0/minicluster-data/ts-1/wal/wals/872a08ec0e5244cabd7ff74f03179648
I20260809 15:36:12.370085 12143 ts_tablet_manager.cc:1939] T 872a08ec0e5244cabd7ff74f03179648 P 2b150c9788e84505a9f96579e7b19e75: Deleting consensus metadata
I20260809 15:36:12.370607 12144 ts_tablet_manager.cc:1905] T 872a08ec0e5244cabd7ff74f03179648 P 13f0c3e0f54b458484b5a99d9b58fc82: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 15:36:12.372496 12144 ts_tablet_manager.cc:1918] T 872a08ec0e5244cabd7ff74f03179648 P 13f0c3e0f54b458484b5a99d9b58fc82: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 15:36:12.372562 12144 log.cc:1196] T 872a08ec0e5244cabd7ff74f03179648 P 13f0c3e0f54b458484b5a99d9b58fc82: Deleting WAL directory at /tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786289732823074-3431-0/minicluster-data/ts-2/wal/wals/872a08ec0e5244cabd7ff74f03179648
I20260809 15:36:12.372852 12144 ts_tablet_manager.cc:1939] T 872a08ec0e5244cabd7ff74f03179648 P 13f0c3e0f54b458484b5a99d9b58fc82: Deleting consensus metadata
I20260809 15:36:12.372802 11544 catalog_manager.cc:4841] TS 2b150c9788e84505a9f96579e7b19e75 (127.3.89.194:34141): tablet 872a08ec0e5244cabd7ff74f03179648 (table test-table [id=d3443ca3a45a442b8480236b4959c786]) successfully deleted
I20260809 15:36:12.373124 12145 ts_tablet_manager.cc:1905] T 872a08ec0e5244cabd7ff74f03179648 P d50738272bb8411c92663d6b4be495d0: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 15:36:12.373458 11545 catalog_manager.cc:4841] TS 13f0c3e0f54b458484b5a99d9b58fc82 (127.3.89.195:43403): tablet 872a08ec0e5244cabd7ff74f03179648 (table test-table [id=d3443ca3a45a442b8480236b4959c786]) successfully deleted
I20260809 15:36:12.375485 12145 ts_tablet_manager.cc:1918] T 872a08ec0e5244cabd7ff74f03179648 P d50738272bb8411c92663d6b4be495d0: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 15:36:12.375561 12145 log.cc:1196] T 872a08ec0e5244cabd7ff74f03179648 P d50738272bb8411c92663d6b4be495d0: Deleting WAL directory at /tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786289732823074-3431-0/minicluster-data/ts-0/wal/wals/872a08ec0e5244cabd7ff74f03179648
I20260809 15:36:12.375818 12145 ts_tablet_manager.cc:1939] T 872a08ec0e5244cabd7ff74f03179648 P d50738272bb8411c92663d6b4be495d0: Deleting consensus metadata
I20260809 15:36:12.376328 11560 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:39674:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260809 15:36:12.376463 11546 catalog_manager.cc:4841] TS d50738272bb8411c92663d6b4be495d0 (127.3.89.193:42041): tablet 872a08ec0e5244cabd7ff74f03179648 (table test-table [id=d3443ca3a45a442b8480236b4959c786]) successfully deleted
W20260809 15:36:12.376753 11546 catalog_manager.cc:4808] TS d50738272bb8411c92663d6b4be495d0 (127.3.89.193:42041): delete failed for tablet 872a08ec0e5244cabd7ff74f03179648 because the tablet was not found. No further retry: Not found: Tablet not found: 872a08ec0e5244cabd7ff74f03179648
W20260809 15:36:12.376753 11560 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260809 15:36:12.381232 11706 tablet_service.cc:1467] Processing CreateTablet for tablet 4ade9c4a7f8e49fb974519d4c02358f9 (DEFAULT_TABLE table=test-table [id=f5dc9600e1514658b58e8ece47ecd599]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 15:36:12.381232 11862 tablet_service.cc:1467] Processing CreateTablet for tablet 4ade9c4a7f8e49fb974519d4c02358f9 (DEFAULT_TABLE table=test-table [id=f5dc9600e1514658b58e8ece47ecd599]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 15:36:12.381335 12016 tablet_service.cc:1467] Processing CreateTablet for tablet 4ade9c4a7f8e49fb974519d4c02358f9 (DEFAULT_TABLE table=test-table [id=f5dc9600e1514658b58e8ece47ecd599]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 15:36:12.381521 11706 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 4ade9c4a7f8e49fb974519d4c02358f9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 15:36:12.381520 11862 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 4ade9c4a7f8e49fb974519d4c02358f9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 15:36:12.381589 12016 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 4ade9c4a7f8e49fb974519d4c02358f9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 15:36:12.383849 12116 tablet_bootstrap.cc:492] T 4ade9c4a7f8e49fb974519d4c02358f9 P 13f0c3e0f54b458484b5a99d9b58fc82: Bootstrap starting.
I20260809 15:36:12.384374 12117 tablet_bootstrap.cc:492] T 4ade9c4a7f8e49fb974519d4c02358f9 P d50738272bb8411c92663d6b4be495d0: Bootstrap starting.
I20260809 15:36:12.384374 12115 tablet_bootstrap.cc:492] T 4ade9c4a7f8e49fb974519d4c02358f9 P 2b150c9788e84505a9f96579e7b19e75: Bootstrap starting.
I20260809 15:36:12.384706 12116 tablet_bootstrap.cc:654] T 4ade9c4a7f8e49fb974519d4c02358f9 P 13f0c3e0f54b458484b5a99d9b58fc82: Neither blocks nor log segments found. Creating new log.
I20260809 15:36:12.385215 12117 tablet_bootstrap.cc:654] T 4ade9c4a7f8e49fb974519d4c02358f9 P d50738272bb8411c92663d6b4be495d0: Neither blocks nor log segments found. Creating new log.
I20260809 15:36:12.385215 12115 tablet_bootstrap.cc:654] T 4ade9c4a7f8e49fb974519d4c02358f9 P 2b150c9788e84505a9f96579e7b19e75: Neither blocks nor log segments found. Creating new log.
I20260809 15:36:12.386490 12115 tablet_bootstrap.cc:492] T 4ade9c4a7f8e49fb974519d4c02358f9 P 2b150c9788e84505a9f96579e7b19e75: No bootstrap required, opened a new log
I20260809 15:36:12.386600 12115 ts_tablet_manager.cc:1397] T 4ade9c4a7f8e49fb974519d4c02358f9 P 2b150c9788e84505a9f96579e7b19e75: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260809 15:36:12.387080 12115 raft_consensus.cc:348] T 4ade9c4a7f8e49fb974519d4c02358f9 P 2b150c9788e84505a9f96579e7b19e75 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d50738272bb8411c92663d6b4be495d0" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 42041 } } peers { permanent_uuid: "2b150c9788e84505a9f96579e7b19e75" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 34141 } } peers { permanent_uuid: "13f0c3e0f54b458484b5a99d9b58fc82" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 43403 } }
I20260809 15:36:12.387151 12116 tablet_bootstrap.cc:492] T 4ade9c4a7f8e49fb974519d4c02358f9 P 13f0c3e0f54b458484b5a99d9b58fc82: No bootstrap required, opened a new log
I20260809 15:36:12.387180 12115 raft_consensus.cc:374] T 4ade9c4a7f8e49fb974519d4c02358f9 P 2b150c9788e84505a9f96579e7b19e75 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 15:36:12.387205 12115 raft_consensus.cc:729] T 4ade9c4a7f8e49fb974519d4c02358f9 P 2b150c9788e84505a9f96579e7b19e75 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2b150c9788e84505a9f96579e7b19e75, State: Initialized, Role: FOLLOWER
I20260809 15:36:12.387221 12116 ts_tablet_manager.cc:1397] T 4ade9c4a7f8e49fb974519d4c02358f9 P 13f0c3e0f54b458484b5a99d9b58fc82: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260809 15:36:12.387346 12115 consensus_queue.cc:260] T 4ade9c4a7f8e49fb974519d4c02358f9 P 2b150c9788e84505a9f96579e7b19e75 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d50738272bb8411c92663d6b4be495d0" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 42041 } } peers { permanent_uuid: "2b150c9788e84505a9f96579e7b19e75" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 34141 } } peers { permanent_uuid: "13f0c3e0f54b458484b5a99d9b58fc82" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 43403 } }
I20260809 15:36:12.387573 12116 raft_consensus.cc:348] T 4ade9c4a7f8e49fb974519d4c02358f9 P 13f0c3e0f54b458484b5a99d9b58fc82 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d50738272bb8411c92663d6b4be495d0" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 42041 } } peers { permanent_uuid: "2b150c9788e84505a9f96579e7b19e75" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 34141 } } peers { permanent_uuid: "13f0c3e0f54b458484b5a99d9b58fc82" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 43403 } }
I20260809 15:36:12.387625 12115 ts_tablet_manager.cc:1428] T 4ade9c4a7f8e49fb974519d4c02358f9 P 2b150c9788e84505a9f96579e7b19e75: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260809 15:36:12.387668 12116 raft_consensus.cc:374] T 4ade9c4a7f8e49fb974519d4c02358f9 P 13f0c3e0f54b458484b5a99d9b58fc82 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 15:36:12.387698 12116 raft_consensus.cc:729] T 4ade9c4a7f8e49fb974519d4c02358f9 P 13f0c3e0f54b458484b5a99d9b58fc82 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 13f0c3e0f54b458484b5a99d9b58fc82, State: Initialized, Role: FOLLOWER
I20260809 15:36:12.387815 12116 consensus_queue.cc:260] T 4ade9c4a7f8e49fb974519d4c02358f9 P 13f0c3e0f54b458484b5a99d9b58fc82 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d50738272bb8411c92663d6b4be495d0" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 42041 } } peers { permanent_uuid: "2b150c9788e84505a9f96579e7b19e75" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 34141 } } peers { permanent_uuid: "13f0c3e0f54b458484b5a99d9b58fc82" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 43403 } }
I20260809 15:36:12.388052 12116 ts_tablet_manager.cc:1428] T 4ade9c4a7f8e49fb974519d4c02358f9 P 13f0c3e0f54b458484b5a99d9b58fc82: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260809 15:36:12.388165 12117 tablet_bootstrap.cc:492] T 4ade9c4a7f8e49fb974519d4c02358f9 P d50738272bb8411c92663d6b4be495d0: No bootstrap required, opened a new log
I20260809 15:36:12.388258 12117 ts_tablet_manager.cc:1397] T 4ade9c4a7f8e49fb974519d4c02358f9 P d50738272bb8411c92663d6b4be495d0: Time spent bootstrapping tablet: real 0.004s user 0.001s sys 0.000s
I20260809 15:36:12.388742 12117 raft_consensus.cc:348] T 4ade9c4a7f8e49fb974519d4c02358f9 P d50738272bb8411c92663d6b4be495d0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d50738272bb8411c92663d6b4be495d0" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 42041 } } peers { permanent_uuid: "2b150c9788e84505a9f96579e7b19e75" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 34141 } } peers { permanent_uuid: "13f0c3e0f54b458484b5a99d9b58fc82" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 43403 } }
I20260809 15:36:12.388885 12117 raft_consensus.cc:374] T 4ade9c4a7f8e49fb974519d4c02358f9 P d50738272bb8411c92663d6b4be495d0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 15:36:12.388916 12117 raft_consensus.cc:729] T 4ade9c4a7f8e49fb974519d4c02358f9 P d50738272bb8411c92663d6b4be495d0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d50738272bb8411c92663d6b4be495d0, State: Initialized, Role: FOLLOWER
I20260809 15:36:12.389045 12117 consensus_queue.cc:260] T 4ade9c4a7f8e49fb974519d4c02358f9 P d50738272bb8411c92663d6b4be495d0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d50738272bb8411c92663d6b4be495d0" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 42041 } } peers { permanent_uuid: "2b150c9788e84505a9f96579e7b19e75" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 34141 } } peers { permanent_uuid: "13f0c3e0f54b458484b5a99d9b58fc82" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 43403 } }
I20260809 15:36:12.389330 12117 ts_tablet_manager.cc:1428] T 4ade9c4a7f8e49fb974519d4c02358f9 P d50738272bb8411c92663d6b4be495d0: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260809 15:36:12.803097 12180 raft_consensus.cc:482] T 4ade9c4a7f8e49fb974519d4c02358f9 P d50738272bb8411c92663d6b4be495d0 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 15:36:12.803231 12180 raft_consensus.cc:504] T 4ade9c4a7f8e49fb974519d4c02358f9 P d50738272bb8411c92663d6b4be495d0 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d50738272bb8411c92663d6b4be495d0" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 42041 } } peers { permanent_uuid: "2b150c9788e84505a9f96579e7b19e75" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 34141 } } peers { permanent_uuid: "13f0c3e0f54b458484b5a99d9b58fc82" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 43403 } }
I20260809 15:36:12.803640 12180 leader_election.cc:288] T 4ade9c4a7f8e49fb974519d4c02358f9 P d50738272bb8411c92663d6b4be495d0 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 2b150c9788e84505a9f96579e7b19e75 (127.3.89.194:34141), 13f0c3e0f54b458484b5a99d9b58fc82 (127.3.89.195:43403)
I20260809 15:36:12.804155 12037 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4ade9c4a7f8e49fb974519d4c02358f9" candidate_uuid: "d50738272bb8411c92663d6b4be495d0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "13f0c3e0f54b458484b5a99d9b58fc82" is_pre_election: true
I20260809 15:36:12.804159 11882 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4ade9c4a7f8e49fb974519d4c02358f9" candidate_uuid: "d50738272bb8411c92663d6b4be495d0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2b150c9788e84505a9f96579e7b19e75" is_pre_election: true
I20260809 15:36:12.804324 12037 raft_consensus.cc:2454] T 4ade9c4a7f8e49fb974519d4c02358f9 P 13f0c3e0f54b458484b5a99d9b58fc82 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d50738272bb8411c92663d6b4be495d0 in term 0.
I20260809 15:36:12.804333 11882 raft_consensus.cc:2454] T 4ade9c4a7f8e49fb974519d4c02358f9 P 2b150c9788e84505a9f96579e7b19e75 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d50738272bb8411c92663d6b4be495d0 in term 0.
I20260809 15:36:12.804723 11654 leader_election.cc:302] T 4ade9c4a7f8e49fb974519d4c02358f9 P d50738272bb8411c92663d6b4be495d0 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2b150c9788e84505a9f96579e7b19e75, d50738272bb8411c92663d6b4be495d0; no voters:
I20260809 15:36:12.804951 12180 raft_consensus.cc:2781] T 4ade9c4a7f8e49fb974519d4c02358f9 P d50738272bb8411c92663d6b4be495d0 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260809 15:36:12.805029 12180 raft_consensus.cc:482] T 4ade9c4a7f8e49fb974519d4c02358f9 P d50738272bb8411c92663d6b4be495d0 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 15:36:12.805086 12180 raft_consensus.cc:3037] T 4ade9c4a7f8e49fb974519d4c02358f9 P d50738272bb8411c92663d6b4be495d0 [term 0 FOLLOWER]: Advancing to term 1
I20260809 15:36:12.806129 12180 raft_consensus.cc:504] T 4ade9c4a7f8e49fb974519d4c02358f9 P d50738272bb8411c92663d6b4be495d0 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d50738272bb8411c92663d6b4be495d0" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 42041 } } peers { permanent_uuid: "2b150c9788e84505a9f96579e7b19e75" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 34141 } } peers { permanent_uuid: "13f0c3e0f54b458484b5a99d9b58fc82" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 43403 } }
I20260809 15:36:12.806464 12180 leader_election.cc:288] T 4ade9c4a7f8e49fb974519d4c02358f9 P d50738272bb8411c92663d6b4be495d0 [CANDIDATE]: Term 1 election: Requested vote from peers 2b150c9788e84505a9f96579e7b19e75 (127.3.89.194:34141), 13f0c3e0f54b458484b5a99d9b58fc82 (127.3.89.195:43403)
I20260809 15:36:12.806843 11882 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4ade9c4a7f8e49fb974519d4c02358f9" candidate_uuid: "d50738272bb8411c92663d6b4be495d0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2b150c9788e84505a9f96579e7b19e75"
I20260809 15:36:12.806860 12037 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4ade9c4a7f8e49fb974519d4c02358f9" candidate_uuid: "d50738272bb8411c92663d6b4be495d0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "13f0c3e0f54b458484b5a99d9b58fc82"
I20260809 15:36:12.806998 11882 raft_consensus.cc:3037] T 4ade9c4a7f8e49fb974519d4c02358f9 P 2b150c9788e84505a9f96579e7b19e75 [term 0 FOLLOWER]: Advancing to term 1
I20260809 15:36:12.806998 12037 raft_consensus.cc:3037] T 4ade9c4a7f8e49fb974519d4c02358f9 P 13f0c3e0f54b458484b5a99d9b58fc82 [term 0 FOLLOWER]: Advancing to term 1
I20260809 15:36:12.808144 12037 raft_consensus.cc:2454] T 4ade9c4a7f8e49fb974519d4c02358f9 P 13f0c3e0f54b458484b5a99d9b58fc82 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d50738272bb8411c92663d6b4be495d0 in term 1.
I20260809 15:36:12.808151 11882 raft_consensus.cc:2454] T 4ade9c4a7f8e49fb974519d4c02358f9 P 2b150c9788e84505a9f96579e7b19e75 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d50738272bb8411c92663d6b4be495d0 in term 1.
I20260809 15:36:12.808425 11656 leader_election.cc:302] T 4ade9c4a7f8e49fb974519d4c02358f9 P d50738272bb8411c92663d6b4be495d0 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 13f0c3e0f54b458484b5a99d9b58fc82, d50738272bb8411c92663d6b4be495d0; no voters:
I20260809 15:36:12.808624 12180 raft_consensus.cc:2781] T 4ade9c4a7f8e49fb974519d4c02358f9 P d50738272bb8411c92663d6b4be495d0 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 15:36:12.808820 12180 raft_consensus.cc:686] T 4ade9c4a7f8e49fb974519d4c02358f9 P d50738272bb8411c92663d6b4be495d0 [term 1 LEADER]: Becoming Leader. State: Replica: d50738272bb8411c92663d6b4be495d0, State: Running, Role: LEADER
I20260809 15:36:12.808977 12180 consensus_queue.cc:237] T 4ade9c4a7f8e49fb974519d4c02358f9 P d50738272bb8411c92663d6b4be495d0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d50738272bb8411c92663d6b4be495d0" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 42041 } } peers { permanent_uuid: "2b150c9788e84505a9f96579e7b19e75" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 34141 } } peers { permanent_uuid: "13f0c3e0f54b458484b5a99d9b58fc82" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 43403 } }
I20260809 15:36:12.810587 11558 catalog_manager.cc:5495] T 4ade9c4a7f8e49fb974519d4c02358f9 P d50738272bb8411c92663d6b4be495d0 reported cstate change: term changed from 0 to 1, leader changed from <none> to d50738272bb8411c92663d6b4be495d0 (127.3.89.193). New cstate: current_term: 1 leader_uuid: "d50738272bb8411c92663d6b4be495d0" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d50738272bb8411c92663d6b4be495d0" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 42041 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "2b150c9788e84505a9f96579e7b19e75" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 34141 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "13f0c3e0f54b458484b5a99d9b58fc82" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 43403 } health_report { overall_health: UNKNOWN } } }
I20260809 15:36:12.849558 12213 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:36:12.846232 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43596 (local address 127.3.89.193:42041)
0809 15:36:12.846540 (+ 308us) server_negotiation.cc:206] Beginning negotiation
0809 15:36:12.846545 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 15:36:12.846557 (+ 12us) server_negotiation.cc:407] Connection header received
0809 15:36:12.846598 (+ 41us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 15:36:12.846603 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 15:36:12.846647 (+ 44us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 15:36:12.846739 (+ 92us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 15:36:12.847080 (+ 341us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:36:12.847618 (+ 538us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:36:12.848456 (+ 838us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:36:12.848711 (+ 255us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:36:12.848837 (+ 126us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:36:12.849078 (+ 241us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 15:36:12.849177 (+ 99us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 15:36:12.849258 (+ 81us) server_negotiation.cc:1035] Waiting for connection context
0809 15:36:12.849357 (+ 99us) server_negotiation.cc:299] Negotiation successful
0809 15:36:12.849410 (+ 53us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":217,"thread_start_us":117,"threads_started":1}
I20260809 15:36:12.855808 12037 raft_consensus.cc:1261] T 4ade9c4a7f8e49fb974519d4c02358f9 P 13f0c3e0f54b458484b5a99d9b58fc82 [term 1 FOLLOWER]: Refusing update from remote peer d50738272bb8411c92663d6b4be495d0: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 15:36:12.855883 11882 raft_consensus.cc:1261] T 4ade9c4a7f8e49fb974519d4c02358f9 P 2b150c9788e84505a9f96579e7b19e75 [term 1 FOLLOWER]: Refusing update from remote peer d50738272bb8411c92663d6b4be495d0: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 15:36:12.856238 12211 consensus_queue.cc:1035] T 4ade9c4a7f8e49fb974519d4c02358f9 P d50738272bb8411c92663d6b4be495d0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "13f0c3e0f54b458484b5a99d9b58fc82" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 43403 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 15:36:12.856417 12180 consensus_queue.cc:1035] T 4ade9c4a7f8e49fb974519d4c02358f9 P d50738272bb8411c92663d6b4be495d0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2b150c9788e84505a9f96579e7b19e75" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 34141 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 15:36:12.888228 11559 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:39674:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 15:36:12.888403 11559 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:39674:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 15:36:12.890316 11559 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 6a9afd191e4f40c2a679860b4257c4a7: Sending DeleteTablet for 3 replicas of tablet 4ade9c4a7f8e49fb974519d4c02358f9
I20260809 15:36:12.890857 11706 tablet_service.cc:1514] Processing DeleteTablet for tablet 4ade9c4a7f8e49fb974519d4c02358f9 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 15:36:12 UTC) from {username='oryx', principal='oryx/127.3.89.254@KRBTEST.COM'} at 127.0.0.1:43584
I20260809 15:36:12.890864 11862 tablet_service.cc:1514] Processing DeleteTablet for tablet 4ade9c4a7f8e49fb974519d4c02358f9 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 15:36:12 UTC) from {username='oryx', principal='oryx/127.3.89.254@KRBTEST.COM'} at 127.0.0.1:34258
I20260809 15:36:12.891098 12016 tablet_service.cc:1514] Processing DeleteTablet for tablet 4ade9c4a7f8e49fb974519d4c02358f9 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 15:36:12 UTC) from {username='oryx', principal='oryx/127.3.89.254@KRBTEST.COM'} at 127.0.0.1:45132
I20260809 15:36:12.891209 12220 tablet_replica.cc:330] T 4ade9c4a7f8e49fb974519d4c02358f9 P d50738272bb8411c92663d6b4be495d0: stopping tablet replica
I20260809 15:36:12.891350 12220 raft_consensus.cc:2229] T 4ade9c4a7f8e49fb974519d4c02358f9 P d50738272bb8411c92663d6b4be495d0 [term 1 LEADER]: Raft consensus shutting down.
I20260809 15:36:12.891378 12221 tablet_replica.cc:330] T 4ade9c4a7f8e49fb974519d4c02358f9 P 13f0c3e0f54b458484b5a99d9b58fc82: stopping tablet replica
I20260809 15:36:12.891513 12221 raft_consensus.cc:2229] T 4ade9c4a7f8e49fb974519d4c02358f9 P 13f0c3e0f54b458484b5a99d9b58fc82 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 15:36:12.891563 12220 raft_consensus.cc:2258] T 4ade9c4a7f8e49fb974519d4c02358f9 P d50738272bb8411c92663d6b4be495d0 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 15:36:12.891638 12221 raft_consensus.cc:2258] T 4ade9c4a7f8e49fb974519d4c02358f9 P 13f0c3e0f54b458484b5a99d9b58fc82 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 15:36:12.891810 12219 tablet_replica.cc:330] T 4ade9c4a7f8e49fb974519d4c02358f9 P 2b150c9788e84505a9f96579e7b19e75: stopping tablet replica
I20260809 15:36:12.891940 12219 raft_consensus.cc:2229] T 4ade9c4a7f8e49fb974519d4c02358f9 P 2b150c9788e84505a9f96579e7b19e75 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 15:36:12.892057 12219 raft_consensus.cc:2258] T 4ade9c4a7f8e49fb974519d4c02358f9 P 2b150c9788e84505a9f96579e7b19e75 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 15:36:12.892251 12220 ts_tablet_manager.cc:1905] T 4ade9c4a7f8e49fb974519d4c02358f9 P d50738272bb8411c92663d6b4be495d0: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 15:36:12.892477 3431 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskm314YR/build/debug/bin/kudu with pid 11637
I20260809 15:36:12.897759 12219 ts_tablet_manager.cc:1905] T 4ade9c4a7f8e49fb974519d4c02358f9 P 2b150c9788e84505a9f96579e7b19e75: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 15:36:12.897907 12221 ts_tablet_manager.cc:1905] T 4ade9c4a7f8e49fb974519d4c02358f9 P 13f0c3e0f54b458484b5a99d9b58fc82: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 15:36:12.900043 12219 ts_tablet_manager.cc:1918] T 4ade9c4a7f8e49fb974519d4c02358f9 P 2b150c9788e84505a9f96579e7b19e75: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260809 15:36:12.900156 12219 log.cc:1196] T 4ade9c4a7f8e49fb974519d4c02358f9 P 2b150c9788e84505a9f96579e7b19e75: Deleting WAL directory at /tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786289732823074-3431-0/minicluster-data/ts-1/wal/wals/4ade9c4a7f8e49fb974519d4c02358f9
I20260809 15:36:12.900480 12219 ts_tablet_manager.cc:1939] T 4ade9c4a7f8e49fb974519d4c02358f9 P 2b150c9788e84505a9f96579e7b19e75: Deleting consensus metadata
I20260809 15:36:12.901156 11544 catalog_manager.cc:4841] TS 2b150c9788e84505a9f96579e7b19e75 (127.3.89.194:34141): tablet 4ade9c4a7f8e49fb974519d4c02358f9 (table test-table [id=f5dc9600e1514658b58e8ece47ecd599]) successfully deleted
I20260809 15:36:12.901417 12221 ts_tablet_manager.cc:1918] T 4ade9c4a7f8e49fb974519d4c02358f9 P 13f0c3e0f54b458484b5a99d9b58fc82: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260809 15:36:12.901507 12221 log.cc:1196] T 4ade9c4a7f8e49fb974519d4c02358f9 P 13f0c3e0f54b458484b5a99d9b58fc82: Deleting WAL directory at /tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786289732823074-3431-0/minicluster-data/ts-2/wal/wals/4ade9c4a7f8e49fb974519d4c02358f9
I20260809 15:36:12.901809 12221 ts_tablet_manager.cc:1939] T 4ade9c4a7f8e49fb974519d4c02358f9 P 13f0c3e0f54b458484b5a99d9b58fc82: Deleting consensus metadata
I20260809 15:36:12.902665 11545 catalog_manager.cc:4841] TS 13f0c3e0f54b458484b5a99d9b58fc82 (127.3.89.195:43403): tablet 4ade9c4a7f8e49fb974519d4c02358f9 (table test-table [id=f5dc9600e1514658b58e8ece47ecd599]) successfully deleted
W20260809 15:36:12.902678 11546 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.3.89.193:42041 (error 108)
I20260809 15:36:12.902925 3431 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskm314YR/build/debug/bin/kudu with pid 11790
I20260809 15:36:12.909969 12223 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:36:12.909607 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.89.193:42041 (local address 127.0.0.1:43612)
0809 15:36:12.909820 (+ 213us) negotiation.cc:107] Waiting for socket to connect
0809 15:36:12.909886 (+ 66us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.89.193:42041: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":141,"thread_start_us":57,"threads_started":1}
W20260809 15:36:12.910152 11546 catalog_manager.cc:4584] TS d50738272bb8411c92663d6b4be495d0 (127.3.89.193:42041): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 4ade9c4a7f8e49fb974519d4c02358f9: Network error: Client connection negotiation failed: client connection to 127.3.89.193:42041: connect: Connection refused (error 111)
I20260809 15:36:12.915861 3431 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskm314YR/build/debug/bin/kudu with pid 11940
I20260809 15:36:12.923301 3431 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskm314YR/build/debug/bin/kudu with pid 11528
2026-08-09T15:36:12Z chronyd exiting
[ OK ] SecurityITest.TestNonDefaultPrincipal (3490 ms)
[ RUN ] SecurityITest.TestNonExistentPrincipal
Loading random data
Initializing database '/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786289732823074-3431-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 15:36:12 dist-test-slave-2w3w krb5kdc[12226](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 15:36:12 dist-test-slave-2w3w krb5kdc[12226](info): set up 2 sockets
Aug 09 15:36:12 dist-test-slave-2w3w krb5kdc[12226](info): commencing operation
krb5kdc: starting...
W20260809 15:36:14.013170 3431 mini_kdc.cc:121] Time spent starting KDC: real 1.065s user 0.003s sys 0.003s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786289732823074-3431-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786289732823074-3431-0/krb5kdc/test-user.keytab.
Aug 09 15:36:14 dist-test-slave-2w3w krb5kdc[12226](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786289774, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T15:36:14Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T15:36:14Z Disabled control of system clock
WARNING: no policy specified for kudu/127.3.89.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.3.89.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.3.89.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786289732823074-3431-0/krb5kdc/kudu_127.3.89.254.keytab.
Entry for principal kudu/127.3.89.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786289732823074-3431-0/krb5kdc/kudu_127.3.89.254.keytab.
I20260809 15:36:14.137516 3431 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskm314YR/build/debug/bin/kudu
/tmp/dist-test-taskm314YR/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786289732823074-3431-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786289732823074-3431-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786289732823074-3431-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786289732823074-3431-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.3.89.254:45605
--webserver_interface=127.3.89.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786289732823074-3431-0/krb5kdc/kudu_127.3.89.254.keytab
--principal=kudu/127.3.89.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.3.89.212:37877
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.3.89.254:45605
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--principal=oryx with env {KRB5CCNAME=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786289732823074-3431-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786289732823074-3431-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786289732823074-3431-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 15:36:14.246613 12239 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 15:36:14.246852 12239 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 15:36:14.246917 12239 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 15:36:14.250797 12239 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 15:36:14.250849 12239 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 15:36:14.250866 12239 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 15:36:14.250927 12239 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 15:36:14.250952 12239 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 15:36:14.256152 12239 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.3.89.212:37877
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786289732823074-3431-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786289732823074-3431-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.3.89.254:45605
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786289732823074-3431-0/krb5kdc/kudu_127.3.89.254.keytab
--principal=oryx
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.3.89.254:45605
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786289732823074-3431-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.3.89.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.12239
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786289732823074-3431-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 09 Aug 2026 14:46:55 UTC on ccycloud.quasar-yswkmf.root.comops.site
build id 4576540
I20260809 15:36:14.257076 12239 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 15:36:14.258018 12239 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 15:36:14.266055 12247 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 15:36:14.266072 12244 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 15:36:14.266151 12245 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 15:36:14.266199 12239 server_base.cc:1034] running on GCE node
I20260809 15:36:14.266954 12239 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 15:36:14.268008 12239 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 15:36:14.269196 12239 hybrid_clock.cc:648] HybridClock initialized: now 1786289774269180 us; error 32 us; skew 500 ppm
Runtime error: kinit failed: unable to login from keytab: Keytab contains no suitable keys for oryx@KRBTEST.COM
2026-08-09T15:36:14Z chronyd exiting
[ OK ] SecurityITest.TestNonExistentPrincipal (1348 ms)
[ RUN ] SecurityITest.TestMismatchingPrincipals
Loading random data
Initializing database '/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786289732823074-3431-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 15:36:14 dist-test-slave-2w3w krb5kdc[12252](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 15:36:14 dist-test-slave-2w3w krb5kdc[12252](info): set up 2 sockets
Aug 09 15:36:14 dist-test-slave-2w3w krb5kdc[12252](info): commencing operation
krb5kdc: starting...
W20260809 15:36:15.353228 3431 mini_kdc.cc:121] Time spent starting KDC: real 1.056s user 0.000s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786289732823074-3431-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786289732823074-3431-0/krb5kdc/test-user.keytab.
Aug 09 15:36:15 dist-test-slave-2w3w krb5kdc[12252](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786289775, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T15:36:15Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T15:36:15Z Disabled control of system clock
WARNING: no policy specified for kudu/127.3.89.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.3.89.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.3.89.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786289732823074-3431-0/krb5kdc/kudu_127.3.89.254.keytab.
Entry for principal kudu/127.3.89.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786289732823074-3431-0/krb5kdc/kudu_127.3.89.254.keytab.
I20260809 15:36:15.478647 3431 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskm314YR/build/debug/bin/kudu
/tmp/dist-test-taskm314YR/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786289732823074-3431-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786289732823074-3431-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786289732823074-3431-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786289732823074-3431-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.3.89.254:39259
--webserver_interface=127.3.89.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786289732823074-3431-0/krb5kdc/kudu_127.3.89.254.keytab
--principal=kudu/127.3.89.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.3.89.212:36625
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.3.89.254:39259
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786289732823074-3431-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786289732823074-3431-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786289732823074-3431-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 15:36:15.588754 12266 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 15:36:15.589023 12266 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 15:36:15.589063 12266 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 15:36:15.592983 12266 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 15:36:15.593036 12266 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 15:36:15.593056 12266 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 15:36:15.593072 12266 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 15:36:15.593087 12266 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 15:36:15.598166 12266 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.3.89.212:36625
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786289732823074-3431-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786289732823074-3431-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.3.89.254:39259
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786289732823074-3431-0/krb5kdc/kudu_127.3.89.254.keytab
--principal=kudu/127.3.89.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.3.89.254:39259
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786289732823074-3431-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.3.89.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.12266
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786289732823074-3431-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 09 Aug 2026 14:46:55 UTC on ccycloud.quasar-yswkmf.root.comops.site
build id 4576540
I20260809 15:36:15.599133 12266 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 15:36:15.600054 12266 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 15:36:15.608248 12276 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 15:36:15.608363 12275 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 15:36:15.608289 12266 server_base.cc:1034] running on GCE node
W20260809 15:36:15.608295 12278 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 15:36:15.609018 12266 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 15:36:15.610013 12266 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 15:36:15.611182 12266 hybrid_clock.cc:648] HybridClock initialized: now 1786289775611149 us; error 48 us; skew 500 ppm
Aug 09 15:36:15 dist-test-slave-2w3w krb5kdc[12252](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786289775, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.89.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 15:36:15.614022 12266 init.cc:377] Logged in from keytab as kudu/127.3.89.254@KRBTEST.COM (short username kudu)
I20260809 15:36:15.614935 12266 webserver.cc:460] Webserver started at http://127.3.89.254:41537/ using document root <none> and password file <none>
I20260809 15:36:15.615533 12266 fs_manager.cc:359] Metadata directory not provided
I20260809 15:36:15.615597 12266 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 15:36:15.615832 12266 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 15:36:15.617815 12266 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786289732823074-3431-0/minicluster-data/master-0/data/instance:
uuid: "d688f60b25434fa7a4b92f81abbf7e13"
format_stamp: "Formatted at 2026-08-09 15:36:15 on dist-test-slave-2w3w"
server_key: "1fb2abc2707c5666c5dfa8db7b5402e1"
server_key_iv: "c44303679820cf17b9e83852195990fc"
server_key_version: "encryptionkey@0"
I20260809 15:36:15.618400 12266 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786289732823074-3431-0/minicluster-data/master-0/wal/instance:
uuid: "d688f60b25434fa7a4b92f81abbf7e13"
format_stamp: "Formatted at 2026-08-09 15:36:15 on dist-test-slave-2w3w"
server_key: "1fb2abc2707c5666c5dfa8db7b5402e1"
server_key_iv: "c44303679820cf17b9e83852195990fc"
server_key_version: "encryptionkey@0"
I20260809 15:36:15.621984 12266 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.006s sys 0.000s
I20260809 15:36:15.624313 12285 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 15:36:15.625499 12266 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260809 15:36:15.625641 12266 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786289732823074-3431-0/minicluster-data/master-0/data,/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786289732823074-3431-0/minicluster-data/master-0/wal
uuid: "d688f60b25434fa7a4b92f81abbf7e13"
format_stamp: "Formatted at 2026-08-09 15:36:15 on dist-test-slave-2w3w"
server_key: "1fb2abc2707c5666c5dfa8db7b5402e1"
server_key_iv: "c44303679820cf17b9e83852195990fc"
server_key_version: "encryptionkey@0"
I20260809 15:36:15.625752 12266 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786289732823074-3431-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786289732823074-3431-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786289732823074-3431-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 15:36:15.648145 12266 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 15:36:15.651185 12266 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 15:36:15.651414 12266 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 15:36:15.658927 12266 rpc_server.cc:307] RPC server started. Bound to: 127.3.89.254:39259
I20260809 15:36:15.658972 12341 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.3.89.254:39259 every 8 connection(s)
I20260809 15:36:15.659940 12266 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786289732823074-3431-0/minicluster-data/master-0/data/info.pb
I20260809 15:36:15.662583 12342 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 15:36:15.664943 3431 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskm314YR/build/debug/bin/kudu as pid 12266
I20260809 15:36:15.665081 3431 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786289732823074-3431-0/minicluster-data/master-0/wal/instance
I20260809 15:36:15.665359 3431 external_mini_cluster.cc:1428] Setting key 359881e85a567c4ceff582f1517e28cb
I20260809 15:36:15.668900 12342 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d688f60b25434fa7a4b92f81abbf7e13: Bootstrap starting.
I20260809 15:36:15.671514 12342 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P d688f60b25434fa7a4b92f81abbf7e13: Neither blocks nor log segments found. Creating new log.
I20260809 15:36:15.672339 12342 log.cc:824] T 00000000000000000000000000000000 P d688f60b25434fa7a4b92f81abbf7e13: Log is configured to *not* fsync() on all Append() calls
I20260809 15:36:15.674340 12342 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d688f60b25434fa7a4b92f81abbf7e13: No bootstrap required, opened a new log
Aug 09 15:36:15 dist-test-slave-2w3w krb5kdc[12252](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786289775, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.3.89.254@KRBTEST.COM
I20260809 15:36:15.678522 12342 raft_consensus.cc:348] T 00000000000000000000000000000000 P d688f60b25434fa7a4b92f81abbf7e13 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d688f60b25434fa7a4b92f81abbf7e13" member_type: VOTER last_known_addr { host: "127.3.89.254" port: 39259 } }
I20260809 15:36:15.678778 12342 raft_consensus.cc:374] T 00000000000000000000000000000000 P d688f60b25434fa7a4b92f81abbf7e13 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 15:36:15.678857 12342 raft_consensus.cc:729] T 00000000000000000000000000000000 P d688f60b25434fa7a4b92f81abbf7e13 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d688f60b25434fa7a4b92f81abbf7e13, State: Initialized, Role: FOLLOWER
I20260809 15:36:15.679330 12342 consensus_queue.cc:260] T 00000000000000000000000000000000 P d688f60b25434fa7a4b92f81abbf7e13 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d688f60b25434fa7a4b92f81abbf7e13" member_type: VOTER last_known_addr { host: "127.3.89.254" port: 39259 } }
I20260809 15:36:15.679490 12342 raft_consensus.cc:388] T 00000000000000000000000000000000 P d688f60b25434fa7a4b92f81abbf7e13 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 15:36:15.679558 12342 raft_consensus.cc:482] T 00000000000000000000000000000000 P d688f60b25434fa7a4b92f81abbf7e13 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 15:36:15.679651 12342 raft_consensus.cc:3037] T 00000000000000000000000000000000 P d688f60b25434fa7a4b92f81abbf7e13 [term 0 FOLLOWER]: Advancing to term 1
I20260809 15:36:15.680747 12342 raft_consensus.cc:504] T 00000000000000000000000000000000 P d688f60b25434fa7a4b92f81abbf7e13 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d688f60b25434fa7a4b92f81abbf7e13" member_type: VOTER last_known_addr { host: "127.3.89.254" port: 39259 } }
I20260809 15:36:15.681084 12342 leader_election.cc:302] T 00000000000000000000000000000000 P d688f60b25434fa7a4b92f81abbf7e13 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: d688f60b25434fa7a4b92f81abbf7e13; no voters:
I20260809 15:36:15.681427 12342 leader_election.cc:288] T 00000000000000000000000000000000 P d688f60b25434fa7a4b92f81abbf7e13 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 15:36:15.681562 12347 raft_consensus.cc:2781] T 00000000000000000000000000000000 P d688f60b25434fa7a4b92f81abbf7e13 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 15:36:15.681547 12345 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:36:15.666781 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45756 (local address 127.3.89.254:39259)
0809 15:36:15.667236 (+ 455us) server_negotiation.cc:206] Beginning negotiation
0809 15:36:15.667254 (+ 18us) server_negotiation.cc:399] Waiting for connection header
0809 15:36:15.667298 (+ 44us) server_negotiation.cc:407] Connection header received
0809 15:36:15.668138 (+ 840us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 15:36:15.668162 (+ 24us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 15:36:15.668554 (+ 392us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 15:36:15.668916 (+ 362us) server_negotiation.cc:226] Negotiated authn=SASL
0809 15:36:15.669718 (+ 802us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:36:15.671237 (+ 1519us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:36:15.672140 (+ 903us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:36:15.672883 (+ 743us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:36:15.673003 (+ 120us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:36:15.675629 (+ 2626us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 15:36:15.675636 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 15:36:15.675651 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 15:36:15.675682 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0809 15:36:15.678311 (+ 2629us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 15:36:15.678744 (+ 433us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 15:36:15.678751 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 15:36:15.678756 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0809 15:36:15.678941 (+ 185us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 15:36:15.679298 (+ 357us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 15:36:15.679301 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 15:36:15.679303 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 15:36:15.679812 (+ 509us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 15:36:15.680017 (+ 205us) server_negotiation.cc:1035] Waiting for connection context
0809 15:36:15.680276 (+ 259us) server_negotiation.cc:299] Negotiation successful
0809 15:36:15.680580 (+ 304us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":311,"thread_start_us":167,"threads_started":1}
I20260809 15:36:15.681821 12347 raft_consensus.cc:686] T 00000000000000000000000000000000 P d688f60b25434fa7a4b92f81abbf7e13 [term 1 LEADER]: Becoming Leader. State: Replica: d688f60b25434fa7a4b92f81abbf7e13, State: Running, Role: LEADER
I20260809 15:36:15.682133 12347 consensus_queue.cc:237] T 00000000000000000000000000000000 P d688f60b25434fa7a4b92f81abbf7e13 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d688f60b25434fa7a4b92f81abbf7e13" member_type: VOTER last_known_addr { host: "127.3.89.254" port: 39259 } }
I20260809 15:36:15.682636 12342 sys_catalog.cc:564] T 00000000000000000000000000000000 P d688f60b25434fa7a4b92f81abbf7e13 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 15:36:15.683387 12349 sys_catalog.cc:455] T 00000000000000000000000000000000 P d688f60b25434fa7a4b92f81abbf7e13 [sys.catalog]: SysCatalogTable state changed. Reason: New leader d688f60b25434fa7a4b92f81abbf7e13. Latest consensus state: current_term: 1 leader_uuid: "d688f60b25434fa7a4b92f81abbf7e13" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d688f60b25434fa7a4b92f81abbf7e13" member_type: VOTER last_known_addr { host: "127.3.89.254" port: 39259 } } }
I20260809 15:36:15.683527 12349 sys_catalog.cc:458] T 00000000000000000000000000000000 P d688f60b25434fa7a4b92f81abbf7e13 [sys.catalog]: This master's current role is: LEADER
I20260809 15:36:15.683838 12348 sys_catalog.cc:455] T 00000000000000000000000000000000 P d688f60b25434fa7a4b92f81abbf7e13 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "d688f60b25434fa7a4b92f81abbf7e13" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d688f60b25434fa7a4b92f81abbf7e13" member_type: VOTER last_known_addr { host: "127.3.89.254" port: 39259 } } }
I20260809 15:36:15.683933 12348 sys_catalog.cc:458] T 00000000000000000000000000000000 P d688f60b25434fa7a4b92f81abbf7e13 [sys.catalog]: This master's current role is: LEADER
W20260809 15:36:15.687469 12363 catalog_manager.cc:1558] T 00000000000000000000000000000000 P d688f60b25434fa7a4b92f81abbf7e13: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260809 15:36:15.687572 12363 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260809 15:36:15.687649 12357 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 15:36:15.688357 12357 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 15:36:15.693778 12357 catalog_manager.cc:1347] Generated new cluster ID: 7bd03808278a4a25899f082727bcaad8
I20260809 15:36:15.693842 12357 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 15:36:15.702077 12357 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 15:36:15.703320 12357 catalog_manager.cc:1504] Loading token signing keys...
I20260809 15:36:15.716060 12357 catalog_manager.cc:5868] T 00000000000000000000000000000000 P d688f60b25434fa7a4b92f81abbf7e13: Generated new TSK 0
I20260809 15:36:15.716881 12357 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.3.89.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.3.89.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.3.89.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786289732823074-3431-0/krb5kdc/kudu_127.3.89.193.keytab.
Entry for principal kudu/127.3.89.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786289732823074-3431-0/krb5kdc/kudu_127.3.89.193.keytab.
I20260809 15:36:15.753263 3431 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskm314YR/build/debug/bin/kudu
/tmp/dist-test-taskm314YR/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786289732823074-3431-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786289732823074-3431-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786289732823074-3431-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786289732823074-3431-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.3.89.193:0
--local_ip_for_outbound_sockets=127.3.89.193
--webserver_interface=127.3.89.193
--webserver_port=0
--tserver_master_addrs=127.3.89.254:39259
--keytab_file=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786289732823074-3431-0/krb5kdc/kudu_127.3.89.193.keytab
--principal=kudu/127.3.89.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.3.89.212:36625
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786289732823074-3431-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786289732823074-3431-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786289732823074-3431-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 15:36:15.866108 12370 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 15:36:15.866372 12370 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 15:36:15.866417 12370 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 15:36:15.871176 12370 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 15:36:15.871240 12370 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 15:36:15.871301 12370 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.3.89.193
I20260809 15:36:15.876554 12370 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.3.89.212:36625
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786289732823074-3431-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786289732823074-3431-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786289732823074-3431-0/krb5kdc/kudu_127.3.89.193.keytab
--principal=kudu/127.3.89.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.3.89.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786289732823074-3431-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.3.89.193
--webserver_port=0
--tserver_master_addrs=127.3.89.254:39259
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.12370
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.3.89.193
--log_dir=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786289732823074-3431-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 09 Aug 2026 14:46:55 UTC on ccycloud.quasar-yswkmf.root.comops.site
build id 4576540
I20260809 15:36:15.877550 12370 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 15:36:15.878492 12370 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 15:36:15.888196 12378 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 15:36:15.888252 12375 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 15:36:15.888255 12376 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 15:36:15.888458 12370 server_base.cc:1034] running on GCE node
I20260809 15:36:15.888938 12370 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 15:36:15.889635 12370 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 15:36:15.890844 12370 hybrid_clock.cc:648] HybridClock initialized: now 1786289775890831 us; error 27 us; skew 500 ppm
Aug 09 15:36:15 dist-test-slave-2w3w krb5kdc[12252](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786289775, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.89.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 15:36:15.896966 12370 init.cc:377] Logged in from keytab as kudu/127.3.89.193@KRBTEST.COM (short username kudu)
I20260809 15:36:15.897877 12370 webserver.cc:460] Webserver started at http://127.3.89.193:35625/ using document root <none> and password file <none>
I20260809 15:36:15.898438 12370 fs_manager.cc:359] Metadata directory not provided
I20260809 15:36:15.898532 12370 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 15:36:15.898770 12370 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 15:36:15.900910 12370 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786289732823074-3431-0/minicluster-data/ts-0/data/instance:
uuid: "497d7be5d5b942f2876f2626ee142d94"
format_stamp: "Formatted at 2026-08-09 15:36:15 on dist-test-slave-2w3w"
server_key: "c2e0744681a429d647e3b40a60bd79be"
server_key_iv: "1b225fb24420d4205bcfebcb4ca6808f"
server_key_version: "encryptionkey@0"
I20260809 15:36:15.901497 12370 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786289732823074-3431-0/minicluster-data/ts-0/wal/instance:
uuid: "497d7be5d5b942f2876f2626ee142d94"
format_stamp: "Formatted at 2026-08-09 15:36:15 on dist-test-slave-2w3w"
server_key: "c2e0744681a429d647e3b40a60bd79be"
server_key_iv: "1b225fb24420d4205bcfebcb4ca6808f"
server_key_version: "encryptionkey@0"
I20260809 15:36:15.905104 12370 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.005s sys 0.000s
I20260809 15:36:15.907442 12385 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 15:36:15.908617 12370 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260809 15:36:15.908759 12370 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786289732823074-3431-0/minicluster-data/ts-0/data,/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786289732823074-3431-0/minicluster-data/ts-0/wal
uuid: "497d7be5d5b942f2876f2626ee142d94"
format_stamp: "Formatted at 2026-08-09 15:36:15 on dist-test-slave-2w3w"
server_key: "c2e0744681a429d647e3b40a60bd79be"
server_key_iv: "1b225fb24420d4205bcfebcb4ca6808f"
server_key_version: "encryptionkey@0"
I20260809 15:36:15.908867 12370 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786289732823074-3431-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786289732823074-3431-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786289732823074-3431-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 15:36:15.920723 12370 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 15:36:15.923421 12370 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 15:36:15.923606 12370 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 15:36:15.924147 12370 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 15:36:15.924906 12370 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 15:36:15.924975 12370 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 15:36:15.925046 12370 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 15:36:15.925089 12370 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 15:36:15.933917 12370 rpc_server.cc:307] RPC server started. Bound to: 127.3.89.193:44911
I20260809 15:36:15.933945 12505 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.3.89.193:44911 every 8 connection(s)
I20260809 15:36:15.934754 12370 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786289732823074-3431-0/minicluster-data/ts-0/data/info.pb
I20260809 15:36:15.940099 3431 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskm314YR/build/debug/bin/kudu as pid 12370
I20260809 15:36:15.940227 3431 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786289732823074-3431-0/minicluster-data/ts-0/wal/instance
I20260809 15:36:15.940559 3431 external_mini_cluster.cc:1428] Setting key e8ca5e6cab8e03fc6dc99e204a975394
Aug 09 15:36:15 dist-test-slave-2w3w krb5kdc[12252](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786289775, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.89.193@KRBTEST.COM for kudu/127.3.89.254@KRBTEST.COM
I20260809 15:36:15.948640 12345 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:36:15.936424 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.89.193:49313 (local address 127.3.89.254:39259)
0809 15:36:15.936582 (+ 158us) server_negotiation.cc:206] Beginning negotiation
0809 15:36:15.936587 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 15:36:15.937538 (+ 951us) server_negotiation.cc:407] Connection header received
0809 15:36:15.938330 (+ 792us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 15:36:15.938332 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 15:36:15.938378 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 15:36:15.938453 (+ 75us) server_negotiation.cc:226] Negotiated authn=SASL
0809 15:36:15.939589 (+ 1136us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:36:15.940307 (+ 718us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:36:15.941471 (+ 1164us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:36:15.941836 (+ 365us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:36:15.941948 (+ 112us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:36:15.944621 (+ 2673us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 15:36:15.944628 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 15:36:15.944633 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 15:36:15.944663 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0809 15:36:15.946167 (+ 1504us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 15:36:15.946804 (+ 637us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 15:36:15.946806 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 15:36:15.946807 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 15:36:15.946881 (+ 74us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 15:36:15.947310 (+ 429us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 15:36:15.947312 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 15:36:15.947313 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 15:36:15.947487 (+ 174us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 15:36:15.947566 (+ 79us) server_negotiation.cc:1035] Waiting for connection context
0809 15:36:15.948250 (+ 684us) server_negotiation.cc:299] Negotiation successful
0809 15:36:15.948395 (+ 145us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":61}
I20260809 15:36:15.949258 12508 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:36:15.936790 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.89.254:39259 (local address 127.3.89.193:49313)
0809 15:36:15.937394 (+ 604us) negotiation.cc:107] Waiting for socket to connect
0809 15:36:15.937433 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0809 15:36:15.938155 (+ 722us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 15:36:15.938617 (+ 462us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 15:36:15.938623 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 15:36:15.938956 (+ 333us) client_negotiation.cc:189] Negotiated authn=SASL
0809 15:36:15.939467 (+ 511us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:36:15.939481 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:36:15.940437 (+ 956us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:36:15.940439 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:36:15.941355 (+ 916us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:36:15.941365 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:36:15.942014 (+ 649us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:36:15.942020 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:36:15.942212 (+ 192us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:36:15.942789 (+ 577us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 15:36:15.942814 (+ 25us) client_negotiation.cc:656] calling sasl_client_start()
0809 15:36:15.944448 (+ 1634us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 15:36:15.946292 (+ 1844us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 15:36:15.946300 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 15:36:15.946315 (+ 15us) client_negotiation.cc:760] calling sasl_client_step()
0809 15:36:15.946675 (+ 360us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 15:36:15.947000 (+ 325us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 15:36:15.947007 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 15:36:15.947012 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0809 15:36:15.947210 (+ 198us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 15:36:15.947600 (+ 390us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 15:36:15.947609 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 15:36:15.947956 (+ 347us) client_negotiation.cc:769] Sending connection context
0809 15:36:15.948157 (+ 201us) client_negotiation.cc:240] Negotiation successful
0809 15:36:15.948423 (+ 266us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":432,"thread_start_us":107,"threads_started":1}
I20260809 15:36:15.950418 12506 heartbeater.cc:347] Connected to a master server at 127.3.89.254:39259
I20260809 15:36:15.950683 12506 heartbeater.cc:464] Registering TS with master...
I20260809 15:36:15.951278 12506 heartbeater.cc:511] Master 127.3.89.254:39259 requested a full tablet report, sending...
WARNING: no policy specified for kudu/127.3.89.194@KRBTEST.COM; defaulting to no policy
I20260809 15:36:15.952844 12302 ts_manager.cc:194] Registered new tserver with Master: 497d7be5d5b942f2876f2626ee142d94 (127.3.89.193:44911)
I20260809 15:36:15.954051 12302 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.3.89.193@KRBTEST.COM'} at 127.3.89.193:49313
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.3.89.194@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.3.89.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786289732823074-3431-0/krb5kdc/kudu_127.3.89.194.keytab.
Entry for principal kudu/127.3.89.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786289732823074-3431-0/krb5kdc/kudu_127.3.89.194.keytab.
I20260809 15:36:15.970729 3431 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskm314YR/build/debug/bin/kudu
/tmp/dist-test-taskm314YR/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786289732823074-3431-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786289732823074-3431-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786289732823074-3431-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786289732823074-3431-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.3.89.194:0
--local_ip_for_outbound_sockets=127.3.89.194
--webserver_interface=127.3.89.194
--webserver_port=0
--tserver_master_addrs=127.3.89.254:39259
--keytab_file=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786289732823074-3431-0/krb5kdc/kudu_127.3.89.194.keytab
--principal=kudu/127.3.89.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.3.89.212:36625
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786289732823074-3431-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786289732823074-3431-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786289732823074-3431-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 15:36:16.080111 12511 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 15:36:16.080415 12511 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 15:36:16.080493 12511 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 15:36:16.084882 12511 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 15:36:16.084975 12511 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 15:36:16.085069 12511 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.3.89.194
I20260809 15:36:16.090431 12511 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.3.89.212:36625
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786289732823074-3431-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786289732823074-3431-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786289732823074-3431-0/krb5kdc/kudu_127.3.89.194.keytab
--principal=kudu/127.3.89.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.3.89.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786289732823074-3431-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.3.89.194
--webserver_port=0
--tserver_master_addrs=127.3.89.254:39259
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.12511
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.3.89.194
--log_dir=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786289732823074-3431-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 09 Aug 2026 14:46:55 UTC on ccycloud.quasar-yswkmf.root.comops.site
build id 4576540
I20260809 15:36:16.091585 12511 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 15:36:16.092600 12511 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 15:36:16.102259 12519 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 15:36:16.102252 12516 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 15:36:16.102252 12517 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 15:36:16.102317 12511 server_base.cc:1034] running on GCE node
I20260809 15:36:16.103034 12511 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 15:36:16.103667 12511 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 15:36:16.104841 12511 hybrid_clock.cc:648] HybridClock initialized: now 1786289776104803 us; error 50 us; skew 500 ppm
Aug 09 15:36:16 dist-test-slave-2w3w krb5kdc[12252](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786289776, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.89.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 15:36:16.107885 12511 init.cc:377] Logged in from keytab as kudu/127.3.89.194@KRBTEST.COM (short username kudu)
I20260809 15:36:16.108810 12511 webserver.cc:460] Webserver started at http://127.3.89.194:36557/ using document root <none> and password file <none>
I20260809 15:36:16.109375 12511 fs_manager.cc:359] Metadata directory not provided
I20260809 15:36:16.109437 12511 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 15:36:16.109655 12511 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 15:36:16.111786 12511 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786289732823074-3431-0/minicluster-data/ts-1/data/instance:
uuid: "f35abea463a14a14a2fc83658b8150ab"
format_stamp: "Formatted at 2026-08-09 15:36:16 on dist-test-slave-2w3w"
server_key: "32c28d7c6764dffd8d9a3871e07720b5"
server_key_iv: "df471ae9c572149b2133556982aaa21d"
server_key_version: "encryptionkey@0"
I20260809 15:36:16.112365 12511 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786289732823074-3431-0/minicluster-data/ts-1/wal/instance:
uuid: "f35abea463a14a14a2fc83658b8150ab"
format_stamp: "Formatted at 2026-08-09 15:36:16 on dist-test-slave-2w3w"
server_key: "32c28d7c6764dffd8d9a3871e07720b5"
server_key_iv: "df471ae9c572149b2133556982aaa21d"
server_key_version: "encryptionkey@0"
I20260809 15:36:16.116091 12511 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.000s sys 0.004s
I20260809 15:36:16.118486 12527 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 15:36:16.119916 12511 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.001s
I20260809 15:36:16.120036 12511 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786289732823074-3431-0/minicluster-data/ts-1/data,/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786289732823074-3431-0/minicluster-data/ts-1/wal
uuid: "f35abea463a14a14a2fc83658b8150ab"
format_stamp: "Formatted at 2026-08-09 15:36:16 on dist-test-slave-2w3w"
server_key: "32c28d7c6764dffd8d9a3871e07720b5"
server_key_iv: "df471ae9c572149b2133556982aaa21d"
server_key_version: "encryptionkey@0"
I20260809 15:36:16.120139 12511 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786289732823074-3431-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786289732823074-3431-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786289732823074-3431-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 15:36:16.131399 12511 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 15:36:16.134109 12511 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 15:36:16.134310 12511 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 15:36:16.134902 12511 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 15:36:16.135814 12511 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 15:36:16.135871 12511 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 15:36:16.135939 12511 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 15:36:16.135979 12511 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 15:36:16.145012 12511 rpc_server.cc:307] RPC server started. Bound to: 127.3.89.194:45397
I20260809 15:36:16.145056 12646 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.3.89.194:45397 every 8 connection(s)
I20260809 15:36:16.145871 12511 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786289732823074-3431-0/minicluster-data/ts-1/data/info.pb
I20260809 15:36:16.147375 3431 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskm314YR/build/debug/bin/kudu as pid 12511
I20260809 15:36:16.147528 3431 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786289732823074-3431-0/minicluster-data/ts-1/wal/instance
I20260809 15:36:16.147867 3431 external_mini_cluster.cc:1428] Setting key 18e8a7564d4ef5d7a7b0125bca5d0a9f
Aug 09 15:36:16 dist-test-slave-2w3w krb5kdc[12252](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786289776, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.89.194@KRBTEST.COM for kudu/127.3.89.254@KRBTEST.COM
WARNING: no policy specified for kudu/127.3.89.195@KRBTEST.COM; defaulting to no policy
I20260809 15:36:16.160590 12345 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:36:16.147649 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.89.194:53129 (local address 127.3.89.254:39259)
0809 15:36:16.147806 (+ 157us) server_negotiation.cc:206] Beginning negotiation
0809 15:36:16.147811 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 15:36:16.148987 (+ 1176us) server_negotiation.cc:407] Connection header received
0809 15:36:16.149951 (+ 964us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 15:36:16.149956 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 15:36:16.150014 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 15:36:16.150104 (+ 90us) server_negotiation.cc:226] Negotiated authn=SASL
0809 15:36:16.151270 (+ 1166us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:36:16.151932 (+ 662us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:36:16.153693 (+ 1761us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:36:16.153914 (+ 221us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:36:16.154005 (+ 91us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:36:16.156843 (+ 2838us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 15:36:16.156849 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 15:36:16.156852 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 15:36:16.156880 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0809 15:36:16.158604 (+ 1724us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 15:36:16.159145 (+ 541us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 15:36:16.159152 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 15:36:16.159156 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 15:36:16.159253 (+ 97us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 15:36:16.159583 (+ 330us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 15:36:16.159585 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 15:36:16.159586 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 15:36:16.159764 (+ 178us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 15:36:16.159839 (+ 75us) server_negotiation.cc:1035] Waiting for connection context
0809 15:36:16.160365 (+ 526us) server_negotiation.cc:299] Negotiation successful
0809 15:36:16.160474 (+ 109us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":59}
I20260809 15:36:16.161415 12649 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:36:16.148121 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.89.254:39259 (local address 127.3.89.194:53129)
0809 15:36:16.148830 (+ 709us) negotiation.cc:107] Waiting for socket to connect
0809 15:36:16.148884 (+ 54us) client_negotiation.cc:174] Beginning negotiation
0809 15:36:16.149746 (+ 862us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 15:36:16.150243 (+ 497us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 15:36:16.150249 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 15:36:16.150582 (+ 333us) client_negotiation.cc:189] Negotiated authn=SASL
0809 15:36:16.151132 (+ 550us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:36:16.151146 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:36:16.152062 (+ 916us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:36:16.152067 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:36:16.153549 (+ 1482us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:36:16.153564 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:36:16.154078 (+ 514us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:36:16.154086 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:36:16.154325 (+ 239us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:36:16.154953 (+ 628us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 15:36:16.154996 (+ 43us) client_negotiation.cc:656] calling sasl_client_start()
0809 15:36:16.156664 (+ 1668us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 15:36:16.158723 (+ 2059us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 15:36:16.158728 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 15:36:16.158738 (+ 10us) client_negotiation.cc:760] calling sasl_client_step()
0809 15:36:16.159043 (+ 305us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 15:36:16.159349 (+ 306us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 15:36:16.159351 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 15:36:16.159353 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 15:36:16.159481 (+ 128us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 15:36:16.159854 (+ 373us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 15:36:16.159858 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 15:36:16.160076 (+ 218us) client_negotiation.cc:769] Sending connection context
0809 15:36:16.160219 (+ 143us) client_negotiation.cc:240] Negotiation successful
0809 15:36:16.160523 (+ 304us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":502,"thread_start_us":103,"threads_started":1}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.3.89.195@KRBTEST.COM" created.
I20260809 15:36:16.162648 12647 heartbeater.cc:347] Connected to a master server at 127.3.89.254:39259
I20260809 15:36:16.162894 12647 heartbeater.cc:464] Registering TS with master...
I20260809 15:36:16.163388 12647 heartbeater.cc:511] Master 127.3.89.254:39259 requested a full tablet report, sending...
I20260809 15:36:16.164384 12302 ts_manager.cc:194] Registered new tserver with Master: f35abea463a14a14a2fc83658b8150ab (127.3.89.194:45397)
I20260809 15:36:16.165043 12302 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.3.89.194@KRBTEST.COM'} at 127.3.89.194:53129
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.3.89.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786289732823074-3431-0/krb5kdc/kudu_127.3.89.195.keytab.
Entry for principal kudu/127.3.89.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786289732823074-3431-0/krb5kdc/kudu_127.3.89.195.keytab.
I20260809 15:36:16.178421 3431 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskm314YR/build/debug/bin/kudu
/tmp/dist-test-taskm314YR/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786289732823074-3431-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786289732823074-3431-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786289732823074-3431-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786289732823074-3431-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.3.89.195:0
--local_ip_for_outbound_sockets=127.3.89.195
--webserver_interface=127.3.89.195
--webserver_port=0
--tserver_master_addrs=127.3.89.254:39259
--keytab_file=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786289732823074-3431-0/krb5kdc/kudu_127.3.89.195.keytab
--principal=kudu/127.3.89.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.3.89.212:36625
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786289732823074-3431-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786289732823074-3431-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786289732823074-3431-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 15:36:16.297428 12652 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 15:36:16.297688 12652 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 15:36:16.297730 12652 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 15:36:16.301735 12652 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 15:36:16.301788 12652 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 15:36:16.301847 12652 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.3.89.195
I20260809 15:36:16.306931 12652 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.3.89.212:36625
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786289732823074-3431-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786289732823074-3431-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786289732823074-3431-0/krb5kdc/kudu_127.3.89.195.keytab
--principal=kudu/127.3.89.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.3.89.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786289732823074-3431-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.3.89.195
--webserver_port=0
--tserver_master_addrs=127.3.89.254:39259
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.12652
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.3.89.195
--log_dir=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786289732823074-3431-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 09 Aug 2026 14:46:55 UTC on ccycloud.quasar-yswkmf.root.comops.site
build id 4576540
I20260809 15:36:16.307888 12652 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 15:36:16.308802 12652 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 15:36:16.318284 12660 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 15:36:16.318284 12658 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 15:36:16.318310 12657 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 15:36:16.318863 12652 server_base.cc:1034] running on GCE node
I20260809 15:36:16.319254 12652 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 15:36:16.319834 12652 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 15:36:16.321012 12652 hybrid_clock.cc:648] HybridClock initialized: now 1786289776321000 us; error 25 us; skew 500 ppm
Aug 09 15:36:16 dist-test-slave-2w3w krb5kdc[12252](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786289776, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.89.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 15:36:16.323715 12652 init.cc:377] Logged in from keytab as kudu/127.3.89.195@KRBTEST.COM (short username kudu)
I20260809 15:36:16.324656 12652 webserver.cc:460] Webserver started at http://127.3.89.195:36965/ using document root <none> and password file <none>
I20260809 15:36:16.325203 12652 fs_manager.cc:359] Metadata directory not provided
I20260809 15:36:16.325322 12652 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 15:36:16.325549 12652 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 15:36:16.327574 12652 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786289732823074-3431-0/minicluster-data/ts-2/data/instance:
uuid: "b6be997136984ae7970d33e01fa3c6b3"
format_stamp: "Formatted at 2026-08-09 15:36:16 on dist-test-slave-2w3w"
server_key: "1a3ff563cc6cc322476c34ec76f652b3"
server_key_iv: "c2ff8731258873956b1d7760190c4cd6"
server_key_version: "encryptionkey@0"
I20260809 15:36:16.328133 12652 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786289732823074-3431-0/minicluster-data/ts-2/wal/instance:
uuid: "b6be997136984ae7970d33e01fa3c6b3"
format_stamp: "Formatted at 2026-08-09 15:36:16 on dist-test-slave-2w3w"
server_key: "1a3ff563cc6cc322476c34ec76f652b3"
server_key_iv: "c2ff8731258873956b1d7760190c4cd6"
server_key_version: "encryptionkey@0"
I20260809 15:36:16.331838 12652 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.003s sys 0.001s
I20260809 15:36:16.334079 12668 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 15:36:16.335376 12652 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260809 15:36:16.335493 12652 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786289732823074-3431-0/minicluster-data/ts-2/data,/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786289732823074-3431-0/minicluster-data/ts-2/wal
uuid: "b6be997136984ae7970d33e01fa3c6b3"
format_stamp: "Formatted at 2026-08-09 15:36:16 on dist-test-slave-2w3w"
server_key: "1a3ff563cc6cc322476c34ec76f652b3"
server_key_iv: "c2ff8731258873956b1d7760190c4cd6"
server_key_version: "encryptionkey@0"
I20260809 15:36:16.335593 12652 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786289732823074-3431-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786289732823074-3431-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786289732823074-3431-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 15:36:16.351727 12652 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 15:36:16.354907 12652 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 15:36:16.355159 12652 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 15:36:16.355737 12652 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 15:36:16.356544 12652 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 15:36:16.356611 12652 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 15:36:16.356679 12652 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 15:36:16.356722 12652 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 15:36:16.365262 12652 rpc_server.cc:307] RPC server started. Bound to: 127.3.89.195:45331
I20260809 15:36:16.365336 12792 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.3.89.195:45331 every 8 connection(s)
I20260809 15:36:16.366140 12652 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786289732823074-3431-0/minicluster-data/ts-2/data/info.pb
Aug 09 15:36:16 dist-test-slave-2w3w krb5kdc[12252](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786289776, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.89.195@KRBTEST.COM for kudu/127.3.89.254@KRBTEST.COM
I20260809 15:36:16.375155 3431 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskm314YR/build/debug/bin/kudu as pid 12652
I20260809 15:36:16.375289 3431 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786289732823074-3431-0/minicluster-data/ts-2/wal/instance
I20260809 15:36:16.375633 3431 external_mini_cluster.cc:1428] Setting key 3015df49e646e9086d461ec65cdc7899
I20260809 15:36:16.379241 12345 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:36:16.367978 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.89.195:42495 (local address 127.3.89.254:39259)
0809 15:36:16.368115 (+ 137us) server_negotiation.cc:206] Beginning negotiation
0809 15:36:16.368119 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 15:36:16.369098 (+ 979us) server_negotiation.cc:407] Connection header received
0809 15:36:16.369882 (+ 784us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 15:36:16.369885 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 15:36:16.369929 (+ 44us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 15:36:16.370008 (+ 79us) server_negotiation.cc:226] Negotiated authn=SASL
0809 15:36:16.371115 (+ 1107us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:36:16.371654 (+ 539us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:36:16.372745 (+ 1091us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:36:16.372970 (+ 225us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:36:16.373056 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:36:16.375537 (+ 2481us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 15:36:16.375542 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 15:36:16.375544 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 15:36:16.375575 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0809 15:36:16.377327 (+ 1752us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 15:36:16.377880 (+ 553us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 15:36:16.377884 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 15:36:16.377886 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 15:36:16.377985 (+ 99us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 15:36:16.378299 (+ 314us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 15:36:16.378302 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 15:36:16.378303 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 15:36:16.378472 (+ 169us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 15:36:16.378549 (+ 77us) server_negotiation.cc:1035] Waiting for connection context
0809 15:36:16.379019 (+ 470us) server_negotiation.cc:299] Negotiation successful
0809 15:36:16.379128 (+ 109us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":52}
I20260809 15:36:16.379976 12795 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:36:16.368342 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.89.254:39259 (local address 127.3.89.195:42495)
0809 15:36:16.368948 (+ 606us) negotiation.cc:107] Waiting for socket to connect
0809 15:36:16.368999 (+ 51us) client_negotiation.cc:174] Beginning negotiation
0809 15:36:16.369714 (+ 715us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 15:36:16.370160 (+ 446us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 15:36:16.370165 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 15:36:16.370488 (+ 323us) client_negotiation.cc:189] Negotiated authn=SASL
0809 15:36:16.370917 (+ 429us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:36:16.370930 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:36:16.371763 (+ 833us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:36:16.371766 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:36:16.372629 (+ 863us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:36:16.372638 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:36:16.373101 (+ 463us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:36:16.373104 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:36:16.373244 (+ 140us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:36:16.373723 (+ 479us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 15:36:16.373738 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0809 15:36:16.375305 (+ 1567us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 15:36:16.377451 (+ 2146us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 15:36:16.377455 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 15:36:16.377472 (+ 17us) client_negotiation.cc:760] calling sasl_client_step()
0809 15:36:16.377776 (+ 304us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 15:36:16.378082 (+ 306us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 15:36:16.378085 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 15:36:16.378086 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0809 15:36:16.378213 (+ 127us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 15:36:16.378563 (+ 350us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 15:36:16.378568 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 15:36:16.378769 (+ 201us) client_negotiation.cc:769] Sending connection context
0809 15:36:16.378917 (+ 148us) client_negotiation.cc:240] Negotiation successful
0809 15:36:16.379221 (+ 304us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":436,"thread_start_us":113,"threads_started":1}
I20260809 15:36:16.380998 12793 heartbeater.cc:347] Connected to a master server at 127.3.89.254:39259
I20260809 15:36:16.381227 12793 heartbeater.cc:464] Registering TS with master...
I20260809 15:36:16.381669 12793 heartbeater.cc:511] Master 127.3.89.254:39259 requested a full tablet report, sending...
I20260809 15:36:16.382627 12302 ts_manager.cc:194] Registered new tserver with Master: b6be997136984ae7970d33e01fa3c6b3 (127.3.89.195:45331)
I20260809 15:36:16.383224 12302 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.3.89.195@KRBTEST.COM'} at 127.3.89.195:42495
I20260809 15:36:16.389453 3431 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
WARNING: no policy specified for oryx/127.3.89.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.3.89.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.3.89.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786289732823074-3431-0/krb5kdc/oryx_127.3.89.193.keytab.
Entry for principal oryx/127.3.89.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786289732823074-3431-0/krb5kdc/oryx_127.3.89.193.keytab.
WARNING: no policy specified for oryx/127.3.89.194@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.3.89.194@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.3.89.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786289732823074-3431-0/krb5kdc/oryx_127.3.89.194.keytab.
Entry for principal oryx/127.3.89.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786289732823074-3431-0/krb5kdc/oryx_127.3.89.194.keytab.
WARNING: no policy specified for oryx/127.3.89.195@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.3.89.195@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.3.89.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786289732823074-3431-0/krb5kdc/oryx_127.3.89.195.keytab.
Entry for principal oryx/127.3.89.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786289732823074-3431-0/krb5kdc/oryx_127.3.89.195.keytab.
I20260809 15:36:16.466041 3431 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskm314YR/build/debug/bin/kudu with pid 12370
I20260809 15:36:16.472666 3431 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskm314YR/build/debug/bin/kudu with pid 12511
I20260809 15:36:16.478826 3431 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskm314YR/build/debug/bin/kudu with pid 12652
I20260809 15:36:16.484982 3431 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskm314YR/build/debug/bin/kudu with pid 12266
I20260809 15:36:16.492889 3431 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskm314YR/build/debug/bin/kudu
/tmp/dist-test-taskm314YR/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786289732823074-3431-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786289732823074-3431-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786289732823074-3431-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786289732823074-3431-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.3.89.254:39259
--webserver_interface=127.3.89.254
--webserver_port=41537
--keytab_file=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786289732823074-3431-0/krb5kdc/kudu_127.3.89.254.keytab
--principal=kudu/127.3.89.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.3.89.212:36625
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.3.89.254:39259
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786289732823074-3431-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786289732823074-3431-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786289732823074-3431-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 15:36:16.601657 12803 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 15:36:16.601938 12803 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 15:36:16.602032 12803 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 15:36:16.605988 12803 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 15:36:16.606071 12803 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 15:36:16.606114 12803 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 15:36:16.606181 12803 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 15:36:16.606222 12803 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 15:36:16.611557 12803 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.3.89.212:36625
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786289732823074-3431-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786289732823074-3431-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.3.89.254:39259
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786289732823074-3431-0/krb5kdc/kudu_127.3.89.254.keytab
--principal=kudu/127.3.89.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.3.89.254:39259
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786289732823074-3431-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.3.89.254
--webserver_port=41537
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.12803
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786289732823074-3431-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 09 Aug 2026 14:46:55 UTC on ccycloud.quasar-yswkmf.root.comops.site
build id 4576540
I20260809 15:36:16.612602 12803 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 15:36:16.613602 12803 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 15:36:16.622049 12809 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 15:36:16.622088 12808 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 15:36:16.622083 12811 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 15:36:16.622319 12803 server_base.cc:1034] running on GCE node
I20260809 15:36:16.622839 12803 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 15:36:16.623905 12803 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 15:36:16.625075 12803 hybrid_clock.cc:648] HybridClock initialized: now 1786289776625058 us; error 31 us; skew 500 ppm
Aug 09 15:36:16 dist-test-slave-2w3w krb5kdc[12252](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786289776, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.89.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 15:36:16.627681 12803 init.cc:377] Logged in from keytab as kudu/127.3.89.254@KRBTEST.COM (short username kudu)
I20260809 15:36:16.628562 12803 webserver.cc:460] Webserver started at http://127.3.89.254:41537/ using document root <none> and password file <none>
I20260809 15:36:16.629073 12803 fs_manager.cc:359] Metadata directory not provided
I20260809 15:36:16.629146 12803 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 15:36:16.632803 12803 fs_manager.cc:709] Time spent opening directory manager: real 0.002s user 0.003s sys 0.000s
I20260809 15:36:16.634789 12819 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 15:36:16.636004 12803 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260809 15:36:16.636132 12803 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786289732823074-3431-0/minicluster-data/master-0/data,/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786289732823074-3431-0/minicluster-data/master-0/wal
uuid: "d688f60b25434fa7a4b92f81abbf7e13"
format_stamp: "Formatted at 2026-08-09 15:36:15 on dist-test-slave-2w3w"
server_key: "1fb2abc2707c5666c5dfa8db7b5402e1"
server_key_iv: "c44303679820cf17b9e83852195990fc"
server_key_version: "encryptionkey@0"
I20260809 15:36:16.636653 12803 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786289732823074-3431-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786289732823074-3431-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786289732823074-3431-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 15:36:16.654627 12803 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 15:36:16.657359 12803 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 15:36:16.657557 12803 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 15:36:16.664785 12803 rpc_server.cc:307] RPC server started. Bound to: 127.3.89.254:39259
I20260809 15:36:16.664824 12874 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.3.89.254:39259 every 8 connection(s)
I20260809 15:36:16.665819 12803 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786289732823074-3431-0/minicluster-data/master-0/data/info.pb
I20260809 15:36:16.669317 3431 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskm314YR/build/debug/bin/kudu as pid 12803
I20260809 15:36:16.669772 3431 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskm314YR/build/debug/bin/kudu
/tmp/dist-test-taskm314YR/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786289732823074-3431-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786289732823074-3431-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786289732823074-3431-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786289732823074-3431-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.3.89.193:44911
--local_ip_for_outbound_sockets=127.3.89.193
--tserver_master_addrs=127.3.89.254:39259
--webserver_port=35625
--webserver_interface=127.3.89.193
--keytab_file=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786289732823074-3431-0/krb5kdc/kudu_127.3.89.193.keytab
--principal=kudu/127.3.89.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.3.89.212:36625
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--principal=oryx/127.3.89.193
--keytab_file=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786289732823074-3431-0/krb5kdc/oryx_127.3.89.193.keytab with env {KRB5CCNAME=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786289732823074-3431-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786289732823074-3431-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786289732823074-3431-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260809 15:36:16.670086 12875 sys_catalog.cc:263] Verifying existing consensus state
I20260809 15:36:16.672102 12875 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d688f60b25434fa7a4b92f81abbf7e13: Bootstrap starting.
I20260809 15:36:16.695906 12875 log.cc:824] T 00000000000000000000000000000000 P d688f60b25434fa7a4b92f81abbf7e13: Log is configured to *not* fsync() on all Append() calls
I20260809 15:36:16.704022 12875 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d688f60b25434fa7a4b92f81abbf7e13: Bootstrap replayed 1/1 log segments. Stats: ops{read=4 overwritten=0 applied=4 ignored=0} inserts{seen=3 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260809 15:36:16.704437 12875 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d688f60b25434fa7a4b92f81abbf7e13: Bootstrap complete.
I20260809 15:36:16.710172 12875 raft_consensus.cc:348] T 00000000000000000000000000000000 P d688f60b25434fa7a4b92f81abbf7e13 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d688f60b25434fa7a4b92f81abbf7e13" member_type: VOTER last_known_addr { host: "127.3.89.254" port: 39259 } }
I20260809 15:36:16.710672 12875 raft_consensus.cc:729] T 00000000000000000000000000000000 P d688f60b25434fa7a4b92f81abbf7e13 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: d688f60b25434fa7a4b92f81abbf7e13, State: Initialized, Role: FOLLOWER
I20260809 15:36:16.711217 12875 consensus_queue.cc:260] T 00000000000000000000000000000000 P d688f60b25434fa7a4b92f81abbf7e13 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d688f60b25434fa7a4b92f81abbf7e13" member_type: VOTER last_known_addr { host: "127.3.89.254" port: 39259 } }
I20260809 15:36:16.711364 12875 raft_consensus.cc:388] T 00000000000000000000000000000000 P d688f60b25434fa7a4b92f81abbf7e13 [term 1 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 15:36:16.711431 12875 raft_consensus.cc:482] T 00000000000000000000000000000000 P d688f60b25434fa7a4b92f81abbf7e13 [term 1 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 15:36:16.711522 12875 raft_consensus.cc:3037] T 00000000000000000000000000000000 P d688f60b25434fa7a4b92f81abbf7e13 [term 1 FOLLOWER]: Advancing to term 2
I20260809 15:36:16.714085 12875 raft_consensus.cc:504] T 00000000000000000000000000000000 P d688f60b25434fa7a4b92f81abbf7e13 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d688f60b25434fa7a4b92f81abbf7e13" member_type: VOTER last_known_addr { host: "127.3.89.254" port: 39259 } }
I20260809 15:36:16.714437 12875 leader_election.cc:302] T 00000000000000000000000000000000 P d688f60b25434fa7a4b92f81abbf7e13 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: d688f60b25434fa7a4b92f81abbf7e13; no voters:
I20260809 15:36:16.714830 12875 leader_election.cc:288] T 00000000000000000000000000000000 P d688f60b25434fa7a4b92f81abbf7e13 [CANDIDATE]: Term 2 election: Requested vote from peers
I20260809 15:36:16.715001 12880 raft_consensus.cc:2781] T 00000000000000000000000000000000 P d688f60b25434fa7a4b92f81abbf7e13 [term 2 FOLLOWER]: Leader election won for term 2
I20260809 15:36:16.715291 12880 raft_consensus.cc:686] T 00000000000000000000000000000000 P d688f60b25434fa7a4b92f81abbf7e13 [term 2 LEADER]: Becoming Leader. State: Replica: d688f60b25434fa7a4b92f81abbf7e13, State: Running, Role: LEADER
I20260809 15:36:16.715637 12880 consensus_queue.cc:237] T 00000000000000000000000000000000 P d688f60b25434fa7a4b92f81abbf7e13 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d688f60b25434fa7a4b92f81abbf7e13" member_type: VOTER last_known_addr { host: "127.3.89.254" port: 39259 } }
I20260809 15:36:16.716066 12875 sys_catalog.cc:564] T 00000000000000000000000000000000 P d688f60b25434fa7a4b92f81abbf7e13 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 15:36:16.717196 12882 sys_catalog.cc:455] T 00000000000000000000000000000000 P d688f60b25434fa7a4b92f81abbf7e13 [sys.catalog]: SysCatalogTable state changed. Reason: New leader d688f60b25434fa7a4b92f81abbf7e13. Latest consensus state: current_term: 2 leader_uuid: "d688f60b25434fa7a4b92f81abbf7e13" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d688f60b25434fa7a4b92f81abbf7e13" member_type: VOTER last_known_addr { host: "127.3.89.254" port: 39259 } } }
I20260809 15:36:16.717196 12881 sys_catalog.cc:455] T 00000000000000000000000000000000 P d688f60b25434fa7a4b92f81abbf7e13 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 2 leader_uuid: "d688f60b25434fa7a4b92f81abbf7e13" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d688f60b25434fa7a4b92f81abbf7e13" member_type: VOTER last_known_addr { host: "127.3.89.254" port: 39259 } } }
I20260809 15:36:16.717365 12881 sys_catalog.cc:458] T 00000000000000000000000000000000 P d688f60b25434fa7a4b92f81abbf7e13 [sys.catalog]: This master's current role is: LEADER
I20260809 15:36:16.717365 12882 sys_catalog.cc:458] T 00000000000000000000000000000000 P d688f60b25434fa7a4b92f81abbf7e13 [sys.catalog]: This master's current role is: LEADER
I20260809 15:36:16.717816 12889 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 15:36:16.721340 12889 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 15:36:16.721993 12889 catalog_manager.cc:1259] Loaded cluster ID: 7bd03808278a4a25899f082727bcaad8
I20260809 15:36:16.722057 12889 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 15:36:16.724393 12889 catalog_manager.cc:1504] Loading token signing keys...
I20260809 15:36:16.725066 12889 catalog_manager.cc:5879] T 00000000000000000000000000000000 P d688f60b25434fa7a4b92f81abbf7e13: Loaded TSK: 0
I20260809 15:36:16.726261 12889 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260809 15:36:16.797389 12877 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 15:36:16.797673 12877 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 15:36:16.797744 12877 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 15:36:16.802201 12877 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 15:36:16.802285 12877 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 15:36:16.802374 12877 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.3.89.193
I20260809 15:36:16.807649 12877 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.3.89.212:36625
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786289732823074-3431-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786289732823074-3431-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786289732823074-3431-0/krb5kdc/oryx_127.3.89.193.keytab
--principal=oryx/127.3.89.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.3.89.193:44911
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786289732823074-3431-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.3.89.193
--webserver_port=35625
--tserver_master_addrs=127.3.89.254:39259
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.12877
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.3.89.193
--log_dir=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786289732823074-3431-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 09 Aug 2026 14:46:55 UTC on ccycloud.quasar-yswkmf.root.comops.site
build id 4576540
I20260809 15:36:16.808605 12877 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 15:36:16.809552 12877 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 15:36:16.820128 12906 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 15:36:16.820134 12905 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 15:36:16.820371 12909 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 15:36:16.820492 12877 server_base.cc:1034] running on GCE node
I20260809 15:36:16.820972 12877 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 15:36:16.821570 12877 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 15:36:16.822757 12877 hybrid_clock.cc:648] HybridClock initialized: now 1786289776822726 us; error 54 us; skew 500 ppm
Aug 09 15:36:16 dist-test-slave-2w3w krb5kdc[12252](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786289776, etypes {rep=17 tkt=17 ses=17}, oryx/127.3.89.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 15:36:16.825453 12877 init.cc:377] Logged in from keytab as oryx/127.3.89.193@KRBTEST.COM (short username oryx)
I20260809 15:36:16.826293 12877 webserver.cc:460] Webserver started at http://127.3.89.193:35625/ using document root <none> and password file <none>
I20260809 15:36:16.826882 12877 fs_manager.cc:359] Metadata directory not provided
I20260809 15:36:16.826997 12877 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 15:36:16.830790 12877 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.000s sys 0.003s
I20260809 15:36:16.832880 12916 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 15:36:16.834160 12877 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260809 15:36:16.834291 12877 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786289732823074-3431-0/minicluster-data/ts-0/data,/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786289732823074-3431-0/minicluster-data/ts-0/wal
uuid: "497d7be5d5b942f2876f2626ee142d94"
format_stamp: "Formatted at 2026-08-09 15:36:15 on dist-test-slave-2w3w"
server_key: "c2e0744681a429d647e3b40a60bd79be"
server_key_iv: "1b225fb24420d4205bcfebcb4ca6808f"
server_key_version: "encryptionkey@0"
I20260809 15:36:16.834864 12877 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786289732823074-3431-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786289732823074-3431-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786289732823074-3431-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 15:36:16.855545 12877 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 15:36:16.858816 12877 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 15:36:16.859068 12877 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 15:36:16.859665 12877 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 15:36:16.860548 12877 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 15:36:16.860617 12877 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 15:36:16.860694 12877 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 15:36:16.860737 12877 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 15:36:16.869626 12877 rpc_server.cc:307] RPC server started. Bound to: 127.3.89.193:44911
I20260809 15:36:16.869655 13035 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.3.89.193:44911 every 8 connection(s)
I20260809 15:36:16.870579 12877 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786289732823074-3431-0/minicluster-data/ts-0/data/info.pb
I20260809 15:36:16.877693 3431 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskm314YR/build/debug/bin/kudu as pid 12877
I20260809 15:36:16.878183 3431 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskm314YR/build/debug/bin/kudu
/tmp/dist-test-taskm314YR/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786289732823074-3431-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786289732823074-3431-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786289732823074-3431-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786289732823074-3431-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.3.89.194:45397
--local_ip_for_outbound_sockets=127.3.89.194
--tserver_master_addrs=127.3.89.254:39259
--webserver_port=36557
--webserver_interface=127.3.89.194
--keytab_file=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786289732823074-3431-0/krb5kdc/kudu_127.3.89.194.keytab
--principal=kudu/127.3.89.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.3.89.212:36625
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--principal=oryx/127.3.89.194
--keytab_file=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786289732823074-3431-0/krb5kdc/oryx_127.3.89.194.keytab with env {KRB5CCNAME=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786289732823074-3431-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786289732823074-3431-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786289732823074-3431-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 09 15:36:16 dist-test-slave-2w3w krb5kdc[12252](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.3.89.193@KRBTEST.COM for oryx/127.3.89.254@KRBTEST.COM, Server not found in Kerberos database
Aug 09 15:36:16 dist-test-slave-2w3w krb5kdc[12252](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.3.89.193@KRBTEST.COM for oryx/127.3.89.254@KRBTEST.COM, Server not found in Kerberos database
W20260809 15:36:16.883944 13039 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 15:36:16.872640 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.89.193:47413 (local address 127.3.89.254:39259)
0809 15:36:16.873172 (+ 532us) server_negotiation.cc:206] Beginning negotiation
0809 15:36:16.873197 (+ 25us) server_negotiation.cc:399] Waiting for connection header
0809 15:36:16.873651 (+ 454us) server_negotiation.cc:407] Connection header received
0809 15:36:16.874743 (+ 1092us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 15:36:16.874754 (+ 11us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 15:36:16.875065 (+ 311us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 15:36:16.875386 (+ 321us) server_negotiation.cc:226] Negotiated authn=SASL
0809 15:36:16.876550 (+ 1164us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:36:16.877523 (+ 973us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:36:16.878852 (+ 1329us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:36:16.879389 (+ 537us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:36:16.879567 (+ 178us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:36:16.883273 (+ 3706us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.3.89.193:47413: BlockingRecv error: recv got EOF from 127.3.89.193:47413 (error 108)
Metrics: {"server-negotiator.queue_time_us":316,"thread_start_us":136,"threads_started":1}
I20260809 15:36:16.884058 13040 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:36:16.872740 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.89.254:39259 (local address 127.3.89.193:47413)
0809 15:36:16.873489 (+ 749us) negotiation.cc:107] Waiting for socket to connect
0809 15:36:16.873530 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0809 15:36:16.874322 (+ 792us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 15:36:16.875474 (+ 1152us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 15:36:16.875482 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 15:36:16.875913 (+ 431us) client_negotiation.cc:189] Negotiated authn=SASL
0809 15:36:16.876361 (+ 448us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:36:16.876375 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:36:16.877668 (+ 1293us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:36:16.877672 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:36:16.878721 (+ 1049us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:36:16.878732 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:36:16.879556 (+ 824us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:36:16.879560 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:36:16.879774 (+ 214us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:36:16.880380 (+ 606us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 15:36:16.880396 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0809 15:36:16.883057 (+ 2661us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.3.89.254:39259: Server oryx/127.3.89.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":574,"thread_start_us":135,"threads_started":1}
W20260809 15:36:16.884642 13037 heartbeater.cc:644] Failed to heartbeat to 127.3.89.254:39259 (0 consecutive failures): Not authorized: Failed to ping master at 127.3.89.254:39259: Client connection negotiation failed: client connection to 127.3.89.254:39259: Server oryx/127.3.89.254@KRBTEST.COM not found in Kerberos database
Aug 09 15:36:16 dist-test-slave-2w3w krb5kdc[12252](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.3.89.193@KRBTEST.COM for oryx/127.3.89.254@KRBTEST.COM, Server not found in Kerberos database
Aug 09 15:36:16 dist-test-slave-2w3w krb5kdc[12252](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.3.89.193@KRBTEST.COM for oryx/127.3.89.254@KRBTEST.COM, Server not found in Kerberos database
I20260809 15:36:16.890236 13040 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:36:16.885125 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.89.254:39259 (local address 127.3.89.193:48303)
0809 15:36:16.885247 (+ 122us) negotiation.cc:107] Waiting for socket to connect
0809 15:36:16.885263 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0809 15:36:16.885343 (+ 80us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 15:36:16.885611 (+ 268us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 15:36:16.885614 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 15:36:16.885769 (+ 155us) client_negotiation.cc:189] Negotiated authn=SASL
0809 15:36:16.885859 (+ 90us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:36:16.885864 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:36:16.886579 (+ 715us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:36:16.886584 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:36:16.887175 (+ 591us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:36:16.887184 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:36:16.887629 (+ 445us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:36:16.887633 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:36:16.887728 (+ 95us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:36:16.888232 (+ 504us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 15:36:16.888243 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0809 15:36:16.890096 (+ 1853us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.3.89.254:39259: Server oryx/127.3.89.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":43}
W20260809 15:36:16.890319 13039 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 15:36:16.885201 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.89.193:48303 (local address 127.3.89.254:39259)
0809 15:36:16.885318 (+ 117us) server_negotiation.cc:206] Beginning negotiation
0809 15:36:16.885322 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 15:36:16.885334 (+ 12us) server_negotiation.cc:407] Connection header received
0809 15:36:16.885453 (+ 119us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 15:36:16.885463 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 15:36:16.885510 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 15:36:16.885595 (+ 85us) server_negotiation.cc:226] Negotiated authn=SASL
0809 15:36:16.885971 (+ 376us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:36:16.886469 (+ 498us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:36:16.887295 (+ 826us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:36:16.887516 (+ 221us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:36:16.887626 (+ 110us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:36:16.890210 (+ 2584us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.3.89.193:48303: BlockingRecv error: recv got EOF from 127.3.89.193:48303 (error 108)
Metrics: {"server-negotiator.queue_time_us":42}
Aug 09 15:36:16 dist-test-slave-2w3w krb5kdc[12252](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.3.89.193@KRBTEST.COM for oryx/127.3.89.254@KRBTEST.COM, Server not found in Kerberos database
Aug 09 15:36:16 dist-test-slave-2w3w krb5kdc[12252](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.3.89.193@KRBTEST.COM for oryx/127.3.89.254@KRBTEST.COM, Server not found in Kerberos database
I20260809 15:36:16.895933 13040 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:36:16.891022 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.89.254:39259 (local address 127.3.89.193:33745)
0809 15:36:16.891260 (+ 238us) negotiation.cc:107] Waiting for socket to connect
0809 15:36:16.891274 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0809 15:36:16.891377 (+ 103us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 15:36:16.891622 (+ 245us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 15:36:16.891625 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 15:36:16.891778 (+ 153us) client_negotiation.cc:189] Negotiated authn=SASL
0809 15:36:16.891843 (+ 65us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:36:16.891847 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:36:16.892542 (+ 695us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:36:16.892547 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:36:16.893060 (+ 513us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:36:16.893067 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:36:16.893513 (+ 446us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:36:16.893516 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:36:16.893595 (+ 79us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:36:16.894050 (+ 455us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 15:36:16.894060 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0809 15:36:16.895799 (+ 1739us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.3.89.254:39259: Server oryx/127.3.89.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":171}
W20260809 15:36:16.896046 13039 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 15:36:16.891128 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.89.193:33745 (local address 127.3.89.254:39259)
0809 15:36:16.891235 (+ 107us) server_negotiation.cc:206] Beginning negotiation
0809 15:36:16.891239 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 15:36:16.891340 (+ 101us) server_negotiation.cc:407] Connection header received
0809 15:36:16.891469 (+ 129us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 15:36:16.891474 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 15:36:16.891524 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 15:36:16.891607 (+ 83us) server_negotiation.cc:226] Negotiated authn=SASL
0809 15:36:16.891952 (+ 345us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:36:16.892433 (+ 481us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:36:16.893193 (+ 760us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:36:16.893401 (+ 208us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:36:16.893509 (+ 108us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:36:16.895918 (+ 2409us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.3.89.193:33745: BlockingRecv error: recv got EOF from 127.3.89.193:33745 (error 108)
Metrics: {"server-negotiator.queue_time_us":41}
W20260809 15:36:16.896256 13037 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260809 15:36:16.990429 13042 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 15:36:16.990670 13042 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 15:36:16.990710 13042 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 15:36:16.994547 13042 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 15:36:16.994603 13042 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 15:36:16.994694 13042 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.3.89.194
I20260809 15:36:16.999914 13042 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.3.89.212:36625
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786289732823074-3431-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786289732823074-3431-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786289732823074-3431-0/krb5kdc/oryx_127.3.89.194.keytab
--principal=oryx/127.3.89.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.3.89.194:45397
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786289732823074-3431-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.3.89.194
--webserver_port=36557
--tserver_master_addrs=127.3.89.254:39259
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.13042
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.3.89.194
--log_dir=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786289732823074-3431-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 09 Aug 2026 14:46:55 UTC on ccycloud.quasar-yswkmf.root.comops.site
build id 4576540
I20260809 15:36:17.000844 13042 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 15:36:17.001814 13042 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 15:36:17.011126 13050 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 15:36:17.011180 13048 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 15:36:17.011291 13042 server_base.cc:1034] running on GCE node
W20260809 15:36:17.011183 13047 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 15:36:17.011780 13042 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 15:36:17.012362 13042 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 15:36:17.013515 13042 hybrid_clock.cc:648] HybridClock initialized: now 1786289777013510 us; error 31 us; skew 500 ppm
Aug 09 15:36:17 dist-test-slave-2w3w krb5kdc[12252](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786289777, etypes {rep=17 tkt=17 ses=17}, oryx/127.3.89.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 15:36:17.016454 13042 init.cc:377] Logged in from keytab as oryx/127.3.89.194@KRBTEST.COM (short username oryx)
I20260809 15:36:17.017351 13042 webserver.cc:460] Webserver started at http://127.3.89.194:36557/ using document root <none> and password file <none>
I20260809 15:36:17.017828 13042 fs_manager.cc:359] Metadata directory not provided
I20260809 15:36:17.017896 13042 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 15:36:17.021550 13042 fs_manager.cc:709] Time spent opening directory manager: real 0.002s user 0.002s sys 0.000s
I20260809 15:36:17.023468 13057 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 15:36:17.024652 13042 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260809 15:36:17.024763 13042 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786289732823074-3431-0/minicluster-data/ts-1/data,/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786289732823074-3431-0/minicluster-data/ts-1/wal
uuid: "f35abea463a14a14a2fc83658b8150ab"
format_stamp: "Formatted at 2026-08-09 15:36:16 on dist-test-slave-2w3w"
server_key: "32c28d7c6764dffd8d9a3871e07720b5"
server_key_iv: "df471ae9c572149b2133556982aaa21d"
server_key_version: "encryptionkey@0"
I20260809 15:36:17.025319 13042 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786289732823074-3431-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786289732823074-3431-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786289732823074-3431-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 15:36:17.046859 13042 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 15:36:17.050326 13042 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 15:36:17.050546 13042 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 15:36:17.051321 13042 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 15:36:17.052423 13042 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 15:36:17.052495 13042 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 15:36:17.052546 13042 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 15:36:17.052570 13042 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 15:36:17.061874 13042 rpc_server.cc:307] RPC server started. Bound to: 127.3.89.194:45397
I20260809 15:36:17.061902 13174 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.3.89.194:45397 every 8 connection(s)
I20260809 15:36:17.062829 13042 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786289732823074-3431-0/minicluster-data/ts-1/data/info.pb
I20260809 15:36:17.065935 3431 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskm314YR/build/debug/bin/kudu as pid 13042
I20260809 15:36:17.066376 3431 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskm314YR/build/debug/bin/kudu
/tmp/dist-test-taskm314YR/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786289732823074-3431-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786289732823074-3431-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786289732823074-3431-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786289732823074-3431-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.3.89.195:45331
--local_ip_for_outbound_sockets=127.3.89.195
--tserver_master_addrs=127.3.89.254:39259
--webserver_port=36965
--webserver_interface=127.3.89.195
--keytab_file=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786289732823074-3431-0/krb5kdc/kudu_127.3.89.195.keytab
--principal=kudu/127.3.89.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.3.89.212:36625
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--principal=oryx/127.3.89.195
--keytab_file=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786289732823074-3431-0/krb5kdc/oryx_127.3.89.195.keytab with env {KRB5CCNAME=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786289732823074-3431-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786289732823074-3431-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786289732823074-3431-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 09 15:36:17 dist-test-slave-2w3w krb5kdc[12252](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.3.89.194@KRBTEST.COM for oryx/127.3.89.254@KRBTEST.COM, Server not found in Kerberos database
Aug 09 15:36:17 dist-test-slave-2w3w krb5kdc[12252](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.3.89.194@KRBTEST.COM for oryx/127.3.89.254@KRBTEST.COM, Server not found in Kerberos database
W20260809 15:36:17.074141 13039 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 15:36:17.064610 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.89.194:45467 (local address 127.3.89.254:39259)
0809 15:36:17.064744 (+ 134us) server_negotiation.cc:206] Beginning negotiation
0809 15:36:17.064747 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0809 15:36:17.065704 (+ 957us) server_negotiation.cc:407] Connection header received
0809 15:36:17.066718 (+ 1014us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 15:36:17.066722 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 15:36:17.066792 (+ 70us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 15:36:17.066909 (+ 117us) server_negotiation.cc:226] Negotiated authn=SASL
0809 15:36:17.068051 (+ 1142us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:36:17.068588 (+ 537us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:36:17.069716 (+ 1128us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:36:17.070109 (+ 393us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:36:17.070233 (+ 124us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:36:17.073968 (+ 3735us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.3.89.194:45467: BlockingRecv error: recv got EOF from 127.3.89.194:45467 (error 108)
Metrics: {"server-negotiator.queue_time_us":47}
I20260809 15:36:17.074589 13177 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:36:17.064918 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.89.254:39259 (local address 127.3.89.194:45467)
0809 15:36:17.065545 (+ 627us) negotiation.cc:107] Waiting for socket to connect
0809 15:36:17.065585 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0809 15:36:17.066398 (+ 813us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 15:36:17.067102 (+ 704us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 15:36:17.067109 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 15:36:17.067453 (+ 344us) client_negotiation.cc:189] Negotiated authn=SASL
0809 15:36:17.067902 (+ 449us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:36:17.067914 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:36:17.068714 (+ 800us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:36:17.068719 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:36:17.069587 (+ 868us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:36:17.069596 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:36:17.070260 (+ 664us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:36:17.070264 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:36:17.070413 (+ 149us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:36:17.071079 (+ 666us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 15:36:17.071095 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0809 15:36:17.073756 (+ 2661us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.3.89.254:39259: Server oryx/127.3.89.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":460,"thread_start_us":114,"threads_started":1}
W20260809 15:36:17.075217 13175 heartbeater.cc:644] Failed to heartbeat to 127.3.89.254:39259 (0 consecutive failures): Not authorized: Failed to ping master at 127.3.89.254:39259: Client connection negotiation failed: client connection to 127.3.89.254:39259: Server oryx/127.3.89.254@KRBTEST.COM not found in Kerberos database
Aug 09 15:36:17 dist-test-slave-2w3w krb5kdc[12252](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.3.89.194@KRBTEST.COM for oryx/127.3.89.254@KRBTEST.COM, Server not found in Kerberos database
Aug 09 15:36:17 dist-test-slave-2w3w krb5kdc[12252](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.3.89.194@KRBTEST.COM for oryx/127.3.89.254@KRBTEST.COM, Server not found in Kerberos database
I20260809 15:36:17.081161 13177 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:36:17.075649 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.89.254:39259 (local address 127.3.89.194:39533)
0809 15:36:17.075758 (+ 109us) negotiation.cc:107] Waiting for socket to connect
0809 15:36:17.075775 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0809 15:36:17.075849 (+ 74us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 15:36:17.076244 (+ 395us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 15:36:17.076246 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 15:36:17.076376 (+ 130us) client_negotiation.cc:189] Negotiated authn=SASL
0809 15:36:17.076470 (+ 94us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:36:17.076475 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:36:17.077573 (+ 1098us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:36:17.077575 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:36:17.078098 (+ 523us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:36:17.078105 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:36:17.078726 (+ 621us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:36:17.078728 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:36:17.078795 (+ 67us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:36:17.079315 (+ 520us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 15:36:17.079326 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0809 15:36:17.081074 (+ 1748us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.3.89.254:39259: Server oryx/127.3.89.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":46}
W20260809 15:36:17.081339 13039 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 15:36:17.075845 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.89.194:39533 (local address 127.3.89.254:39259)
0809 15:36:17.075999 (+ 154us) server_negotiation.cc:206] Beginning negotiation
0809 15:36:17.076006 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 15:36:17.076022 (+ 16us) server_negotiation.cc:407] Connection header received
0809 15:36:17.076069 (+ 47us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 15:36:17.076074 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 15:36:17.076126 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 15:36:17.076223 (+ 97us) server_negotiation.cc:226] Negotiated authn=SASL
0809 15:36:17.076582 (+ 359us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:36:17.077465 (+ 883us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:36:17.078212 (+ 747us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:36:17.078613 (+ 401us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:36:17.078744 (+ 131us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:36:17.081228 (+ 2484us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.3.89.194:39533: BlockingRecv error: recv got EOF from 127.3.89.194:39533 (error 108)
Metrics: {"server-negotiator.queue_time_us":56}
Aug 09 15:36:17 dist-test-slave-2w3w krb5kdc[12252](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.3.89.194@KRBTEST.COM for oryx/127.3.89.254@KRBTEST.COM, Server not found in Kerberos database
Aug 09 15:36:17 dist-test-slave-2w3w krb5kdc[12252](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.3.89.194@KRBTEST.COM for oryx/127.3.89.254@KRBTEST.COM, Server not found in Kerberos database
I20260809 15:36:17.086838 13177 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:36:17.081749 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.89.254:39259 (local address 127.3.89.194:42773)
0809 15:36:17.081943 (+ 194us) negotiation.cc:107] Waiting for socket to connect
0809 15:36:17.081954 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0809 15:36:17.082031 (+ 77us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 15:36:17.082250 (+ 219us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 15:36:17.082252 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 15:36:17.082373 (+ 121us) client_negotiation.cc:189] Negotiated authn=SASL
0809 15:36:17.082438 (+ 65us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:36:17.082442 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:36:17.083438 (+ 996us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:36:17.083442 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:36:17.083940 (+ 498us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:36:17.083947 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:36:17.084485 (+ 538us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:36:17.084488 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:36:17.084545 (+ 57us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:36:17.085004 (+ 459us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 15:36:17.085014 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0809 15:36:17.086754 (+ 1740us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.3.89.254:39259: Server oryx/127.3.89.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":136}
W20260809 15:36:17.087127 13039 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 15:36:17.081841 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.89.194:42773 (local address 127.3.89.254:39259)
0809 15:36:17.081966 (+ 125us) server_negotiation.cc:206] Beginning negotiation
0809 15:36:17.081970 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 15:36:17.082002 (+ 32us) server_negotiation.cc:407] Connection header received
0809 15:36:17.082109 (+ 107us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 15:36:17.082112 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 15:36:17.082151 (+ 39us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 15:36:17.082229 (+ 78us) server_negotiation.cc:226] Negotiated authn=SASL
0809 15:36:17.082537 (+ 308us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:36:17.083342 (+ 805us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:36:17.084051 (+ 709us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:36:17.084381 (+ 330us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:36:17.084468 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:36:17.086906 (+ 2438us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.3.89.194:42773: BlockingRecv error: recv got EOF from 127.3.89.194:42773 (error 108)
Metrics: {"server-negotiator.queue_time_us":51}
W20260809 15:36:17.087138 13175 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260809 15:36:17.179709 13178 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 15:36:17.179947 13178 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 15:36:17.179988 13178 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 15:36:17.183804 13178 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 15:36:17.183851 13178 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 15:36:17.183904 13178 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.3.89.195
I20260809 15:36:17.188871 13178 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.3.89.212:36625
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786289732823074-3431-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786289732823074-3431-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786289732823074-3431-0/krb5kdc/oryx_127.3.89.195.keytab
--principal=oryx/127.3.89.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.3.89.195:45331
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786289732823074-3431-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.3.89.195
--webserver_port=36965
--tserver_master_addrs=127.3.89.254:39259
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.13178
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.3.89.195
--log_dir=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786289732823074-3431-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 09 Aug 2026 14:46:55 UTC on ccycloud.quasar-yswkmf.root.comops.site
build id 4576540
I20260809 15:36:17.189785 13178 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 15:36:17.190687 13178 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 15:36:17.199793 13186 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 15:36:17.199780 13188 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 15:36:17.199793 13184 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 15:36:17.200135 13178 server_base.cc:1034] running on GCE node
I20260809 15:36:17.200507 13178 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 15:36:17.201092 13178 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 15:36:17.202247 13178 hybrid_clock.cc:648] HybridClock initialized: now 1786289777202215 us; error 43 us; skew 500 ppm
Aug 09 15:36:17 dist-test-slave-2w3w krb5kdc[12252](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786289777, etypes {rep=17 tkt=17 ses=17}, oryx/127.3.89.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 15:36:17.204908 13178 init.cc:377] Logged in from keytab as oryx/127.3.89.195@KRBTEST.COM (short username oryx)
I20260809 15:36:17.205701 13178 webserver.cc:460] Webserver started at http://127.3.89.195:36965/ using document root <none> and password file <none>
I20260809 15:36:17.206250 13178 fs_manager.cc:359] Metadata directory not provided
I20260809 15:36:17.206355 13178 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 15:36:17.209977 13178 fs_manager.cc:709] Time spent opening directory manager: real 0.002s user 0.002s sys 0.001s
I20260809 15:36:17.211936 13195 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 15:36:17.213073 13178 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260809 15:36:17.213204 13178 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786289732823074-3431-0/minicluster-data/ts-2/data,/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786289732823074-3431-0/minicluster-data/ts-2/wal
uuid: "b6be997136984ae7970d33e01fa3c6b3"
format_stamp: "Formatted at 2026-08-09 15:36:16 on dist-test-slave-2w3w"
server_key: "1a3ff563cc6cc322476c34ec76f652b3"
server_key_iv: "c2ff8731258873956b1d7760190c4cd6"
server_key_version: "encryptionkey@0"
I20260809 15:36:17.213733 13178 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786289732823074-3431-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786289732823074-3431-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786289732823074-3431-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 15:36:17.235164 13178 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 15:36:17.237759 13178 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 15:36:17.237952 13178 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 15:36:17.238564 13178 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 15:36:17.239490 13178 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 15:36:17.239570 13178 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 15:36:17.239658 13178 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 15:36:17.239710 13178 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 15:36:17.248648 13178 rpc_server.cc:307] RPC server started. Bound to: 127.3.89.195:45331
I20260809 15:36:17.248683 13312 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.3.89.195:45331 every 8 connection(s)
I20260809 15:36:17.249589 13178 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786289732823074-3431-0/minicluster-data/ts-2/data/info.pb
I20260809 15:36:17.253950 3431 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskm314YR/build/debug/bin/kudu as pid 13178
Aug 09 15:36:17 dist-test-slave-2w3w krb5kdc[12252](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.3.89.195@KRBTEST.COM for oryx/127.3.89.254@KRBTEST.COM, Server not found in Kerberos database
Aug 09 15:36:17 dist-test-slave-2w3w krb5kdc[12252](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.3.89.195@KRBTEST.COM for oryx/127.3.89.254@KRBTEST.COM, Server not found in Kerberos database
W20260809 15:36:17.260207 13039 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 15:36:17.251253 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.89.195:33939 (local address 127.3.89.254:39259)
0809 15:36:17.251402 (+ 149us) server_negotiation.cc:206] Beginning negotiation
0809 15:36:17.251407 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 15:36:17.252434 (+ 1027us) server_negotiation.cc:407] Connection header received
0809 15:36:17.253194 (+ 760us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 15:36:17.253197 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 15:36:17.253243 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 15:36:17.253317 (+ 74us) server_negotiation.cc:226] Negotiated authn=SASL
0809 15:36:17.254474 (+ 1157us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:36:17.255185 (+ 711us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:36:17.256333 (+ 1148us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:36:17.256576 (+ 243us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:36:17.256685 (+ 109us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:36:17.260090 (+ 3405us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.3.89.195:33939: BlockingRecv error: recv got EOF from 127.3.89.195:33939 (error 108)
Metrics: {"server-negotiator.queue_time_us":55}
I20260809 15:36:17.260846 13315 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:36:17.251670 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.89.254:39259 (local address 127.3.89.195:33939)
0809 15:36:17.252277 (+ 607us) negotiation.cc:107] Waiting for socket to connect
0809 15:36:17.252327 (+ 50us) client_negotiation.cc:174] Beginning negotiation
0809 15:36:17.253028 (+ 701us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 15:36:17.253468 (+ 440us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 15:36:17.253473 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 15:36:17.253793 (+ 320us) client_negotiation.cc:189] Negotiated authn=SASL
0809 15:36:17.254290 (+ 497us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:36:17.254330 (+ 40us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:36:17.255319 (+ 989us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:36:17.255322 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:36:17.256195 (+ 873us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:36:17.256204 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:36:17.256732 (+ 528us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:36:17.256736 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:36:17.256900 (+ 164us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:36:17.257499 (+ 599us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 15:36:17.257515 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0809 15:36:17.259993 (+ 2478us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.3.89.254:39259: Server oryx/127.3.89.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":440,"thread_start_us":108,"threads_started":1}
W20260809 15:36:17.261356 13313 heartbeater.cc:644] Failed to heartbeat to 127.3.89.254:39259 (0 consecutive failures): Not authorized: Failed to ping master at 127.3.89.254:39259: Client connection negotiation failed: client connection to 127.3.89.254:39259: Server oryx/127.3.89.254@KRBTEST.COM not found in Kerberos database
I20260809 15:36:17.263506 13317 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:36:17.254952 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45772 (local address 127.3.89.254:39259)
0809 15:36:17.255208 (+ 256us) server_negotiation.cc:206] Beginning negotiation
0809 15:36:17.255211 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0809 15:36:17.255222 (+ 11us) server_negotiation.cc:407] Connection header received
0809 15:36:17.255253 (+ 31us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 15:36:17.255255 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 15:36:17.255288 (+ 33us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 15:36:17.255402 (+ 114us) server_negotiation.cc:226] Negotiated authn=SASL
0809 15:36:17.256122 (+ 720us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:36:17.256638 (+ 516us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:36:17.257506 (+ 868us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:36:17.257728 (+ 222us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:36:17.257832 (+ 104us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:36:17.259404 (+ 1572us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 15:36:17.259414 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 15:36:17.259433 (+ 19us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 15:36:17.259462 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0809 15:36:17.261265 (+ 1803us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 15:36:17.261815 (+ 550us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 15:36:17.261823 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 15:36:17.261828 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0809 15:36:17.261984 (+ 156us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 15:36:17.262295 (+ 311us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 15:36:17.262298 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 15:36:17.262300 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 15:36:17.262691 (+ 391us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 15:36:17.262893 (+ 202us) server_negotiation.cc:1035] Waiting for connection context
0809 15:36:17.263131 (+ 238us) server_negotiation.cc:299] Negotiation successful
0809 15:36:17.263340 (+ 209us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":164,"thread_start_us":100,"threads_started":1}
Aug 09 15:36:17 dist-test-slave-2w3w krb5kdc[12252](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.3.89.195@KRBTEST.COM for oryx/127.3.89.254@KRBTEST.COM, Server not found in Kerberos database
Aug 09 15:36:17 dist-test-slave-2w3w krb5kdc[12252](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.3.89.195@KRBTEST.COM for oryx/127.3.89.254@KRBTEST.COM, Server not found in Kerberos database
I20260809 15:36:17.267058 13315 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:36:17.261786 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.89.254:39259 (local address 127.3.89.195:55151)
0809 15:36:17.261980 (+ 194us) negotiation.cc:107] Waiting for socket to connect
0809 15:36:17.261991 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0809 15:36:17.262072 (+ 81us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 15:36:17.262260 (+ 188us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 15:36:17.262266 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 15:36:17.262405 (+ 139us) client_negotiation.cc:189] Negotiated authn=SASL
0809 15:36:17.262490 (+ 85us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:36:17.262495 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:36:17.263395 (+ 900us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:36:17.263397 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:36:17.264142 (+ 745us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:36:17.264151 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:36:17.264583 (+ 432us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:36:17.264586 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:36:17.264675 (+ 89us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:36:17.265107 (+ 432us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 15:36:17.265120 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0809 15:36:17.266909 (+ 1789us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.3.89.254:39259: Server oryx/127.3.89.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":115}
W20260809 15:36:17.267140 13039 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 15:36:17.261890 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.89.195:55151 (local address 127.3.89.254:39259)
0809 15:36:17.262013 (+ 123us) server_negotiation.cc:206] Beginning negotiation
0809 15:36:17.262019 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 15:36:17.262035 (+ 16us) server_negotiation.cc:407] Connection header received
0809 15:36:17.262151 (+ 116us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 15:36:17.262153 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 15:36:17.262183 (+ 30us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 15:36:17.262237 (+ 54us) server_negotiation.cc:226] Negotiated authn=SASL
0809 15:36:17.262595 (+ 358us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:36:17.263293 (+ 698us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:36:17.264268 (+ 975us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:36:17.264473 (+ 205us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:36:17.264582 (+ 109us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:36:17.267035 (+ 2453us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.3.89.195:55151: BlockingRecv error: recv got EOF from 127.3.89.195:55151 (error 108)
Metrics: {"server-negotiator.queue_time_us":51}
Aug 09 15:36:17 dist-test-slave-2w3w krb5kdc[12252](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.3.89.195@KRBTEST.COM for oryx/127.3.89.254@KRBTEST.COM, Server not found in Kerberos database
Aug 09 15:36:17 dist-test-slave-2w3w krb5kdc[12252](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.3.89.195@KRBTEST.COM for oryx/127.3.89.254@KRBTEST.COM, Server not found in Kerberos database
I20260809 15:36:17.271670 13315 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:36:17.267509 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.89.254:39259 (local address 127.3.89.195:54337)
0809 15:36:17.267617 (+ 108us) negotiation.cc:107] Waiting for socket to connect
0809 15:36:17.267630 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0809 15:36:17.267701 (+ 71us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 15:36:17.267895 (+ 194us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 15:36:17.267897 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 15:36:17.268016 (+ 119us) client_negotiation.cc:189] Negotiated authn=SASL
0809 15:36:17.268077 (+ 61us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:36:17.268081 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:36:17.268693 (+ 612us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:36:17.268695 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:36:17.269178 (+ 483us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:36:17.269185 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:36:17.269551 (+ 366us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:36:17.269553 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:36:17.269607 (+ 54us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:36:17.269947 (+ 340us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 15:36:17.269957 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0809 15:36:17.271547 (+ 1590us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.3.89.254:39259: Server oryx/127.3.89.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":46}
W20260809 15:36:17.271745 13039 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 15:36:17.267588 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.89.195:54337 (local address 127.3.89.254:39259)
0809 15:36:17.267715 (+ 127us) server_negotiation.cc:206] Beginning negotiation
0809 15:36:17.267719 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 15:36:17.267754 (+ 35us) server_negotiation.cc:407] Connection header received
0809 15:36:17.267781 (+ 27us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 15:36:17.267782 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 15:36:17.267813 (+ 31us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 15:36:17.267869 (+ 56us) server_negotiation.cc:226] Negotiated authn=SASL
0809 15:36:17.268167 (+ 298us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:36:17.268610 (+ 443us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:36:17.269269 (+ 659us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:36:17.269465 (+ 196us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:36:17.269533 (+ 68us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:36:17.271643 (+ 2110us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.3.89.195:54337: BlockingRecv error: recv got EOF from 127.3.89.195:54337 (error 108)
Metrics: {"server-negotiator.queue_time_us":68}
W20260809 15:36:17.271903 13313 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
Aug 09 15:36:17 dist-test-slave-2w3w krb5kdc[12252](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.3.89.193@KRBTEST.COM for oryx/127.3.89.254@KRBTEST.COM, Server not found in Kerberos database
Aug 09 15:36:17 dist-test-slave-2w3w krb5kdc[12252](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.3.89.193@KRBTEST.COM for oryx/127.3.89.254@KRBTEST.COM, Server not found in Kerberos database
I20260809 15:36:17.902334 13318 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:36:17.896950 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.89.254:39259 (local address 127.3.89.193:33283)
0809 15:36:17.897239 (+ 289us) negotiation.cc:107] Waiting for socket to connect
0809 15:36:17.897253 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0809 15:36:17.897343 (+ 90us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 15:36:17.897604 (+ 261us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 15:36:17.897606 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 15:36:17.897798 (+ 192us) client_negotiation.cc:189] Negotiated authn=SASL
0809 15:36:17.897898 (+ 100us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:36:17.897904 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:36:17.898626 (+ 722us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:36:17.898629 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:36:17.899229 (+ 600us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:36:17.899239 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:36:17.899712 (+ 473us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:36:17.899715 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:36:17.899791 (+ 76us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:36:17.900360 (+ 569us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 15:36:17.900372 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0809 15:36:17.902211 (+ 1839us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.3.89.254:39259: Server oryx/127.3.89.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":184,"thread_start_us":86,"threads_started":1}
W20260809 15:36:17.902537 13320 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 15:36:17.897053 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.89.193:33283 (local address 127.3.89.254:39259)
0809 15:36:17.897296 (+ 243us) server_negotiation.cc:206] Beginning negotiation
0809 15:36:17.897299 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0809 15:36:17.897312 (+ 13us) server_negotiation.cc:407] Connection header received
0809 15:36:17.897433 (+ 121us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 15:36:17.897435 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 15:36:17.897490 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 15:36:17.897567 (+ 77us) server_negotiation.cc:226] Negotiated authn=SASL
0809 15:36:17.898000 (+ 433us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:36:17.898528 (+ 528us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:36:17.899331 (+ 803us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:36:17.899588 (+ 257us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:36:17.899683 (+ 95us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:36:17.902342 (+ 2659us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.3.89.193:33283: BlockingRecv error: recv got EOF from 127.3.89.193:33283 (error 108)
Metrics: {"server-negotiator.queue_time_us":179,"thread_start_us":64,"threads_started":1}
Aug 09 15:36:18 dist-test-slave-2w3w krb5kdc[12252](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.3.89.194@KRBTEST.COM for oryx/127.3.89.254@KRBTEST.COM, Server not found in Kerberos database
Aug 09 15:36:18 dist-test-slave-2w3w krb5kdc[12252](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.3.89.194@KRBTEST.COM for oryx/127.3.89.254@KRBTEST.COM, Server not found in Kerberos database
I20260809 15:36:18.093410 13321 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:36:18.087735 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.89.254:39259 (local address 127.3.89.194:48371)
0809 15:36:18.088056 (+ 321us) negotiation.cc:107] Waiting for socket to connect
0809 15:36:18.088072 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0809 15:36:18.088214 (+ 142us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 15:36:18.088486 (+ 272us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 15:36:18.088489 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 15:36:18.088709 (+ 220us) client_negotiation.cc:189] Negotiated authn=SASL
0809 15:36:18.088802 (+ 93us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:36:18.088807 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:36:18.089582 (+ 775us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:36:18.089588 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:36:18.090169 (+ 581us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:36:18.090177 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:36:18.090647 (+ 470us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:36:18.090651 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:36:18.090738 (+ 87us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:36:18.091283 (+ 545us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 15:36:18.091294 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0809 15:36:18.093288 (+ 1994us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.3.89.254:39259: Server oryx/127.3.89.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":245,"thread_start_us":83,"threads_started":1}
W20260809 15:36:18.093575 13320 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 15:36:18.087864 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.89.194:48371 (local address 127.3.89.254:39259)
0809 15:36:18.088023 (+ 159us) server_negotiation.cc:206] Beginning negotiation
0809 15:36:18.088027 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 15:36:18.088153 (+ 126us) server_negotiation.cc:407] Connection header received
0809 15:36:18.088316 (+ 163us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 15:36:18.088320 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 15:36:18.088372 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 15:36:18.088476 (+ 104us) server_negotiation.cc:226] Negotiated authn=SASL
0809 15:36:18.088928 (+ 452us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:36:18.089468 (+ 540us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:36:18.090294 (+ 826us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:36:18.090530 (+ 236us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:36:18.090643 (+ 113us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:36:18.093412 (+ 2769us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.3.89.194:48371: BlockingRecv error: recv got EOF from 127.3.89.194:48371 (error 108)
Metrics: {"server-negotiator.queue_time_us":59}
Aug 09 15:36:18 dist-test-slave-2w3w krb5kdc[12252](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.3.89.195@KRBTEST.COM for oryx/127.3.89.254@KRBTEST.COM, Server not found in Kerberos database
Aug 09 15:36:18 dist-test-slave-2w3w krb5kdc[12252](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.3.89.195@KRBTEST.COM for oryx/127.3.89.254@KRBTEST.COM, Server not found in Kerberos database
I20260809 15:36:18.278195 13322 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:36:18.272548 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.89.254:39259 (local address 127.3.89.195:42803)
0809 15:36:18.272840 (+ 292us) negotiation.cc:107] Waiting for socket to connect
0809 15:36:18.272852 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0809 15:36:18.272978 (+ 126us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 15:36:18.273259 (+ 281us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 15:36:18.273261 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 15:36:18.273478 (+ 217us) client_negotiation.cc:189] Negotiated authn=SASL
0809 15:36:18.273592 (+ 114us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:36:18.273598 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:36:18.274384 (+ 786us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:36:18.274389 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:36:18.275071 (+ 682us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:36:18.275081 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:36:18.275611 (+ 530us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:36:18.275615 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:36:18.275711 (+ 96us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:36:18.276277 (+ 566us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 15:36:18.276294 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0809 15:36:18.278086 (+ 1792us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.3.89.254:39259: Server oryx/127.3.89.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":207,"thread_start_us":85,"threads_started":1}
W20260809 15:36:18.278342 13320 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 15:36:18.272600 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.89.195:42803 (local address 127.3.89.254:39259)
0809 15:36:18.272769 (+ 169us) server_negotiation.cc:206] Beginning negotiation
0809 15:36:18.272773 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 15:36:18.272931 (+ 158us) server_negotiation.cc:407] Connection header received
0809 15:36:18.273085 (+ 154us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 15:36:18.273089 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 15:36:18.273139 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 15:36:18.273237 (+ 98us) server_negotiation.cc:226] Negotiated authn=SASL
0809 15:36:18.273715 (+ 478us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:36:18.274252 (+ 537us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:36:18.275192 (+ 940us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:36:18.275492 (+ 300us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:36:18.275608 (+ 116us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:36:18.278226 (+ 2618us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.3.89.195:42803: BlockingRecv error: recv got EOF from 127.3.89.195:42803 (error 108)
Metrics: {"server-negotiator.queue_time_us":54}
Aug 09 15:36:18 dist-test-slave-2w3w krb5kdc[12252](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.3.89.193@KRBTEST.COM for oryx/127.3.89.254@KRBTEST.COM, Server not found in Kerberos database
Aug 09 15:36:18 dist-test-slave-2w3w krb5kdc[12252](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.3.89.193@KRBTEST.COM for oryx/127.3.89.254@KRBTEST.COM, Server not found in Kerberos database
I20260809 15:36:18.908979 13324 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:36:18.903178 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.89.254:39259 (local address 127.3.89.193:50219)
0809 15:36:18.903454 (+ 276us) negotiation.cc:107] Waiting for socket to connect
0809 15:36:18.903467 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0809 15:36:18.903557 (+ 90us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 15:36:18.903832 (+ 275us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 15:36:18.903834 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 15:36:18.904054 (+ 220us) client_negotiation.cc:189] Negotiated authn=SASL
0809 15:36:18.904189 (+ 135us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:36:18.904205 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:36:18.905136 (+ 931us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:36:18.905140 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:36:18.905778 (+ 638us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:36:18.905788 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:36:18.906264 (+ 476us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:36:18.906268 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:36:18.906371 (+ 103us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:36:18.906892 (+ 521us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 15:36:18.906903 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0809 15:36:18.908785 (+ 1882us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.3.89.254:39259: Server oryx/127.3.89.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":198,"thread_start_us":94,"threads_started":1}
W20260809 15:36:18.909075 13327 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 15:36:18.903255 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.89.193:50219 (local address 127.3.89.254:39259)
0809 15:36:18.903497 (+ 242us) server_negotiation.cc:206] Beginning negotiation
0809 15:36:18.903500 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0809 15:36:18.903514 (+ 14us) server_negotiation.cc:407] Connection header received
0809 15:36:18.903651 (+ 137us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 15:36:18.903653 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 15:36:18.903696 (+ 43us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 15:36:18.903773 (+ 77us) server_negotiation.cc:226] Negotiated authn=SASL
0809 15:36:18.904322 (+ 549us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:36:18.905022 (+ 700us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:36:18.905904 (+ 882us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:36:18.906147 (+ 243us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:36:18.906257 (+ 110us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:36:18.908926 (+ 2669us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.3.89.193:50219: BlockingRecv error: recv got EOF from 127.3.89.193:50219 (error 108)
Metrics: {"server-negotiator.queue_time_us":164,"thread_start_us":65,"threads_started":1}
Aug 09 15:36:19 dist-test-slave-2w3w krb5kdc[12252](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.3.89.194@KRBTEST.COM for oryx/127.3.89.254@KRBTEST.COM, Server not found in Kerberos database
Aug 09 15:36:19 dist-test-slave-2w3w krb5kdc[12252](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.3.89.194@KRBTEST.COM for oryx/127.3.89.254@KRBTEST.COM, Server not found in Kerberos database
I20260809 15:36:19.100014 13328 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:36:19.094244 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.89.254:39259 (local address 127.3.89.194:57175)
0809 15:36:19.094553 (+ 309us) negotiation.cc:107] Waiting for socket to connect
0809 15:36:19.094567 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0809 15:36:19.094701 (+ 134us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 15:36:19.095193 (+ 492us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 15:36:19.095197 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 15:36:19.095383 (+ 186us) client_negotiation.cc:189] Negotiated authn=SASL
0809 15:36:19.095475 (+ 92us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:36:19.095480 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:36:19.096264 (+ 784us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:36:19.096270 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:36:19.096845 (+ 575us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:36:19.096853 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:36:19.097372 (+ 519us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:36:19.097375 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:36:19.097463 (+ 88us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:36:19.098025 (+ 562us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 15:36:19.098037 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0809 15:36:19.099820 (+ 1783us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.3.89.254:39259: Server oryx/127.3.89.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":223,"thread_start_us":107,"threads_started":1}
W20260809 15:36:19.100044 13327 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 15:36:19.094322 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.89.194:57175 (local address 127.3.89.254:39259)
0809 15:36:19.094496 (+ 174us) server_negotiation.cc:206] Beginning negotiation
0809 15:36:19.094500 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 15:36:19.094643 (+ 143us) server_negotiation.cc:407] Connection header received
0809 15:36:19.094862 (+ 219us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 15:36:19.094866 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 15:36:19.094921 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 15:36:19.095020 (+ 99us) server_negotiation.cc:226] Negotiated authn=SASL
0809 15:36:19.095594 (+ 574us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:36:19.096153 (+ 559us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:36:19.096971 (+ 818us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:36:19.097202 (+ 231us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:36:19.097367 (+ 165us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:36:19.099942 (+ 2575us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.3.89.194:57175: BlockingRecv error: recv got EOF from 127.3.89.194:57175 (error 108)
Metrics: {"server-negotiator.queue_time_us":74}
Aug 09 15:36:19 dist-test-slave-2w3w krb5kdc[12252](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.3.89.195@KRBTEST.COM for oryx/127.3.89.254@KRBTEST.COM, Server not found in Kerberos database
Aug 09 15:36:19 dist-test-slave-2w3w krb5kdc[12252](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.3.89.195@KRBTEST.COM for oryx/127.3.89.254@KRBTEST.COM, Server not found in Kerberos database
I20260809 15:36:19.284538 13329 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:36:19.279114 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.89.254:39259 (local address 127.3.89.195:59255)
0809 15:36:19.279436 (+ 322us) negotiation.cc:107] Waiting for socket to connect
0809 15:36:19.279456 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0809 15:36:19.279543 (+ 87us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 15:36:19.279813 (+ 270us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 15:36:19.279815 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 15:36:19.279996 (+ 181us) client_negotiation.cc:189] Negotiated authn=SASL
0809 15:36:19.280091 (+ 95us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:36:19.280096 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:36:19.280850 (+ 754us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:36:19.280853 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:36:19.281416 (+ 563us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:36:19.281425 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:36:19.281917 (+ 492us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:36:19.281921 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:36:19.282015 (+ 94us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:36:19.282573 (+ 558us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 15:36:19.282586 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0809 15:36:19.284362 (+ 1776us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.3.89.254:39259: Server oryx/127.3.89.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":234,"thread_start_us":146,"threads_started":1}
W20260809 15:36:19.284593 13327 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 15:36:19.279332 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.89.195:59255 (local address 127.3.89.254:39259)
0809 15:36:19.279509 (+ 177us) server_negotiation.cc:206] Beginning negotiation
0809 15:36:19.279515 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 15:36:19.279529 (+ 14us) server_negotiation.cc:407] Connection header received
0809 15:36:19.279639 (+ 110us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 15:36:19.279642 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 15:36:19.279686 (+ 44us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 15:36:19.279759 (+ 73us) server_negotiation.cc:226] Negotiated authn=SASL
0809 15:36:19.280194 (+ 435us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:36:19.280710 (+ 516us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:36:19.281548 (+ 838us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:36:19.281794 (+ 246us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:36:19.281902 (+ 108us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:36:19.284487 (+ 2585us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.3.89.195:59255: BlockingRecv error: recv got EOF from 127.3.89.195:59255 (error 108)
Metrics: {"server-negotiator.queue_time_us":88}
Aug 09 15:36:19 dist-test-slave-2w3w krb5kdc[12252](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.3.89.193@KRBTEST.COM for oryx/127.3.89.254@KRBTEST.COM, Server not found in Kerberos database
Aug 09 15:36:19 dist-test-slave-2w3w krb5kdc[12252](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.3.89.193@KRBTEST.COM for oryx/127.3.89.254@KRBTEST.COM, Server not found in Kerberos database
I20260809 15:36:19.916007 13330 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:36:19.909838 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.89.254:39259 (local address 127.3.89.193:56317)
0809 15:36:19.910144 (+ 306us) negotiation.cc:107] Waiting for socket to connect
0809 15:36:19.910157 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0809 15:36:19.910249 (+ 92us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 15:36:19.910571 (+ 322us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 15:36:19.910573 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 15:36:19.910839 (+ 266us) client_negotiation.cc:189] Negotiated authn=SASL
0809 15:36:19.910942 (+ 103us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:36:19.910949 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:36:19.912057 (+ 1108us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:36:19.912060 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:36:19.912647 (+ 587us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:36:19.912656 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:36:19.913217 (+ 561us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:36:19.913220 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:36:19.913315 (+ 95us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:36:19.913879 (+ 564us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 15:36:19.913900 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0809 15:36:19.915839 (+ 1939us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.3.89.254:39259: Server oryx/127.3.89.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":227,"thread_start_us":77,"threads_started":1}
W20260809 15:36:19.916067 13332 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 15:36:19.910132 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.89.193:56317 (local address 127.3.89.254:39259)
0809 15:36:19.910362 (+ 230us) server_negotiation.cc:206] Beginning negotiation
0809 15:36:19.910365 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0809 15:36:19.910378 (+ 13us) server_negotiation.cc:407] Connection header received
0809 15:36:19.910412 (+ 34us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 15:36:19.910414 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 15:36:19.910453 (+ 39us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 15:36:19.910531 (+ 78us) server_negotiation.cc:226] Negotiated authn=SASL
0809 15:36:19.911129 (+ 598us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:36:19.911915 (+ 786us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:36:19.912777 (+ 862us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:36:19.913094 (+ 317us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:36:19.913212 (+ 118us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:36:19.915946 (+ 2734us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.3.89.193:56317: BlockingRecv error: recv got EOF from 127.3.89.193:56317 (error 108)
Metrics: {"server-negotiator.queue_time_us":166,"thread_start_us":83,"threads_started":1}
Aug 09 15:36:20 dist-test-slave-2w3w krb5kdc[12252](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.3.89.194@KRBTEST.COM for oryx/127.3.89.254@KRBTEST.COM, Server not found in Kerberos database
Aug 09 15:36:20 dist-test-slave-2w3w krb5kdc[12252](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.3.89.194@KRBTEST.COM for oryx/127.3.89.254@KRBTEST.COM, Server not found in Kerberos database
I20260809 15:36:20.106657 13333 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:36:20.100777 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.89.254:39259 (local address 127.3.89.194:36271)
0809 15:36:20.101106 (+ 329us) negotiation.cc:107] Waiting for socket to connect
0809 15:36:20.101118 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0809 15:36:20.101258 (+ 140us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 15:36:20.101535 (+ 277us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 15:36:20.101538 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 15:36:20.101761 (+ 223us) client_negotiation.cc:189] Negotiated authn=SASL
0809 15:36:20.101883 (+ 122us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:36:20.101888 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:36:20.102667 (+ 779us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:36:20.102673 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:36:20.103273 (+ 600us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:36:20.103282 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:36:20.103741 (+ 459us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:36:20.103745 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:36:20.103860 (+ 115us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:36:20.104417 (+ 557us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 15:36:20.104428 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0809 15:36:20.106502 (+ 2074us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.3.89.254:39259: Server oryx/127.3.89.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":250,"thread_start_us":113,"threads_started":1}
W20260809 15:36:20.106817 13332 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 15:36:20.100860 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.89.194:36271 (local address 127.3.89.254:39259)
0809 15:36:20.101028 (+ 168us) server_negotiation.cc:206] Beginning negotiation
0809 15:36:20.101033 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 15:36:20.101199 (+ 166us) server_negotiation.cc:407] Connection header received
0809 15:36:20.101362 (+ 163us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 15:36:20.101366 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 15:36:20.101418 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 15:36:20.101511 (+ 93us) server_negotiation.cc:226] Negotiated authn=SASL
0809 15:36:20.102008 (+ 497us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:36:20.102551 (+ 543us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:36:20.103401 (+ 850us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:36:20.103626 (+ 225us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:36:20.103738 (+ 112us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:36:20.106652 (+ 2914us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.3.89.194:36271: BlockingRecv error: recv got EOF from 127.3.89.194:36271 (error 108)
Metrics: {"server-negotiator.queue_time_us":72}
Aug 09 15:36:20 dist-test-slave-2w3w krb5kdc[12252](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.3.89.195@KRBTEST.COM for oryx/127.3.89.254@KRBTEST.COM, Server not found in Kerberos database
Aug 09 15:36:20 dist-test-slave-2w3w krb5kdc[12252](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.3.89.195@KRBTEST.COM for oryx/127.3.89.254@KRBTEST.COM, Server not found in Kerberos database
I20260809 15:36:20.291157 13334 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:36:20.285450 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.89.254:39259 (local address 127.3.89.195:34521)
0809 15:36:20.285779 (+ 329us) negotiation.cc:107] Waiting for socket to connect
0809 15:36:20.285794 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0809 15:36:20.285899 (+ 105us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 15:36:20.286171 (+ 272us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 15:36:20.286174 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 15:36:20.286366 (+ 192us) client_negotiation.cc:189] Negotiated authn=SASL
0809 15:36:20.286469 (+ 103us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:36:20.286475 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:36:20.287201 (+ 726us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:36:20.287204 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:36:20.287798 (+ 594us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:36:20.287806 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:36:20.288212 (+ 406us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:36:20.288215 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:36:20.288277 (+ 62us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:36:20.288884 (+ 607us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 15:36:20.288896 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0809 15:36:20.290942 (+ 2046us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.3.89.254:39259: Server oryx/127.3.89.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":249,"thread_start_us":119,"threads_started":1}
W20260809 15:36:20.291196 13332 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 15:36:20.285518 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.89.195:34521 (local address 127.3.89.254:39259)
0809 15:36:20.285672 (+ 154us) server_negotiation.cc:206] Beginning negotiation
0809 15:36:20.285677 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 15:36:20.285870 (+ 193us) server_negotiation.cc:407] Connection header received
0809 15:36:20.285979 (+ 109us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 15:36:20.285982 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 15:36:20.286027 (+ 45us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 15:36:20.286097 (+ 70us) server_negotiation.cc:226] Negotiated authn=SASL
0809 15:36:20.286574 (+ 477us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:36:20.287107 (+ 533us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:36:20.287901 (+ 794us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:36:20.288114 (+ 213us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:36:20.288187 (+ 73us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:36:20.291087 (+ 2900us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.3.89.195:34521: BlockingRecv error: recv got EOF from 127.3.89.195:34521 (error 108)
Metrics: {"server-negotiator.queue_time_us":64}
Aug 09 15:36:20 dist-test-slave-2w3w krb5kdc[12252](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.3.89.193@KRBTEST.COM for oryx/127.3.89.254@KRBTEST.COM, Server not found in Kerberos database
Aug 09 15:36:20 dist-test-slave-2w3w krb5kdc[12252](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.3.89.193@KRBTEST.COM for oryx/127.3.89.254@KRBTEST.COM, Server not found in Kerberos database
I20260809 15:36:20.928920 13335 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:36:20.916871 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.89.254:39259 (local address 127.3.89.193:46695)
0809 15:36:20.917213 (+ 342us) negotiation.cc:107] Waiting for socket to connect
0809 15:36:20.917230 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0809 15:36:20.917381 (+ 151us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 15:36:20.917672 (+ 291us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 15:36:20.917675 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 15:36:20.917896 (+ 221us) client_negotiation.cc:189] Negotiated authn=SASL
0809 15:36:20.917993 (+ 97us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:36:20.917998 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:36:20.918871 (+ 873us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:36:20.918876 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:36:20.919450 (+ 574us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:36:20.919465 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:36:20.919922 (+ 457us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:36:20.919925 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:36:20.920009 (+ 84us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:36:20.926843 (+ 6834us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 15:36:20.926855 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0809 15:36:20.928749 (+ 1894us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.3.89.254:39259: Server oryx/127.3.89.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":257,"thread_start_us":106,"threads_started":1}
W20260809 15:36:20.929010 13336 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 15:36:20.916963 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.89.193:46695 (local address 127.3.89.254:39259)
0809 15:36:20.917213 (+ 250us) server_negotiation.cc:206] Beginning negotiation
0809 15:36:20.917219 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 15:36:20.917318 (+ 99us) server_negotiation.cc:407] Connection header received
0809 15:36:20.917496 (+ 178us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 15:36:20.917499 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 15:36:20.917553 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 15:36:20.917649 (+ 96us) server_negotiation.cc:226] Negotiated authn=SASL
0809 15:36:20.918215 (+ 566us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:36:20.918743 (+ 528us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:36:20.919585 (+ 842us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:36:20.919811 (+ 226us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:36:20.919919 (+ 108us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:36:20.928874 (+ 8955us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.3.89.193:46695: BlockingRecv error: recv got EOF from 127.3.89.193:46695 (error 108)
Metrics: {"server-negotiator.queue_time_us":167,"thread_start_us":71,"threads_started":1}
Aug 09 15:36:21 dist-test-slave-2w3w krb5kdc[12252](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.3.89.194@KRBTEST.COM for oryx/127.3.89.254@KRBTEST.COM, Server not found in Kerberos database
Aug 09 15:36:21 dist-test-slave-2w3w krb5kdc[12252](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.3.89.194@KRBTEST.COM for oryx/127.3.89.254@KRBTEST.COM, Server not found in Kerberos database
I20260809 15:36:21.113163 13337 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:36:21.107522 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.89.254:39259 (local address 127.3.89.194:51767)
0809 15:36:21.107866 (+ 344us) negotiation.cc:107] Waiting for socket to connect
0809 15:36:21.107880 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0809 15:36:21.108022 (+ 142us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 15:36:21.108305 (+ 283us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 15:36:21.108308 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 15:36:21.108538 (+ 230us) client_negotiation.cc:189] Negotiated authn=SASL
0809 15:36:21.108633 (+ 95us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:36:21.108639 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:36:21.109423 (+ 784us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:36:21.109429 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:36:21.109995 (+ 566us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:36:21.110004 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:36:21.110477 (+ 473us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:36:21.110481 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:36:21.110574 (+ 93us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:36:21.111130 (+ 556us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 15:36:21.111142 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0809 15:36:21.112989 (+ 1847us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.3.89.254:39259: Server oryx/127.3.89.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":255,"thread_start_us":89,"threads_started":1}
W20260809 15:36:21.113235 13336 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 15:36:21.107646 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.89.194:51767 (local address 127.3.89.254:39259)
0809 15:36:21.107812 (+ 166us) server_negotiation.cc:206] Beginning negotiation
0809 15:36:21.107818 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 15:36:21.107970 (+ 152us) server_negotiation.cc:407] Connection header received
0809 15:36:21.108124 (+ 154us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 15:36:21.108127 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 15:36:21.108188 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 15:36:21.108277 (+ 89us) server_negotiation.cc:226] Negotiated authn=SASL
0809 15:36:21.108762 (+ 485us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:36:21.109307 (+ 545us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:36:21.110123 (+ 816us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:36:21.110353 (+ 230us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:36:21.110467 (+ 114us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:36:21.113125 (+ 2658us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.3.89.194:51767: BlockingRecv error: recv got EOF from 127.3.89.194:51767 (error 108)
Metrics: {"server-negotiator.queue_time_us":65}
Aug 09 15:36:21 dist-test-slave-2w3w krb5kdc[12252](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.3.89.195@KRBTEST.COM for oryx/127.3.89.254@KRBTEST.COM, Server not found in Kerberos database
Aug 09 15:36:21 dist-test-slave-2w3w krb5kdc[12252](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.3.89.195@KRBTEST.COM for oryx/127.3.89.254@KRBTEST.COM, Server not found in Kerberos database
I20260809 15:36:21.297596 13338 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:36:21.292031 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.89.254:39259 (local address 127.3.89.195:54741)
0809 15:36:21.292377 (+ 346us) negotiation.cc:107] Waiting for socket to connect
0809 15:36:21.292391 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0809 15:36:21.292542 (+ 151us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 15:36:21.292822 (+ 280us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 15:36:21.292825 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 15:36:21.293043 (+ 218us) client_negotiation.cc:189] Negotiated authn=SASL
0809 15:36:21.293140 (+ 97us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:36:21.293146 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:36:21.293919 (+ 773us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:36:21.293925 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:36:21.294507 (+ 582us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:36:21.294515 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:36:21.295032 (+ 517us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:36:21.295036 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:36:21.295128 (+ 92us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:36:21.295599 (+ 471us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 15:36:21.295611 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0809 15:36:21.297484 (+ 1873us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.3.89.254:39259: Server oryx/127.3.89.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":238,"thread_start_us":109,"threads_started":1}
W20260809 15:36:21.297746 13336 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 15:36:21.292148 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.89.195:54741 (local address 127.3.89.254:39259)
0809 15:36:21.292319 (+ 171us) server_negotiation.cc:206] Beginning negotiation
0809 15:36:21.292324 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 15:36:21.292467 (+ 143us) server_negotiation.cc:407] Connection header received
0809 15:36:21.292645 (+ 178us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 15:36:21.292649 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 15:36:21.292708 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 15:36:21.292797 (+ 89us) server_negotiation.cc:226] Negotiated authn=SASL
0809 15:36:21.293265 (+ 468us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:36:21.293806 (+ 541us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:36:21.294632 (+ 826us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:36:21.294882 (+ 250us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:36:21.295032 (+ 150us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:36:21.297628 (+ 2596us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.3.89.195:54741: BlockingRecv error: recv got EOF from 127.3.89.195:54741 (error 108)
Metrics: {"server-negotiator.queue_time_us":75}
Aug 09 15:36:21 dist-test-slave-2w3w krb5kdc[12252](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.3.89.193@KRBTEST.COM for oryx/127.3.89.254@KRBTEST.COM, Server not found in Kerberos database
Aug 09 15:36:21 dist-test-slave-2w3w krb5kdc[12252](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.3.89.193@KRBTEST.COM for oryx/127.3.89.254@KRBTEST.COM, Server not found in Kerberos database
I20260809 15:36:21.935756 13339 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:36:21.929735 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.89.254:39259 (local address 127.3.89.193:54467)
0809 15:36:21.930068 (+ 333us) negotiation.cc:107] Waiting for socket to connect
0809 15:36:21.930082 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0809 15:36:21.930241 (+ 159us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 15:36:21.930639 (+ 398us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 15:36:21.930644 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 15:36:21.930989 (+ 345us) client_negotiation.cc:189] Negotiated authn=SASL
0809 15:36:21.931119 (+ 130us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:36:21.931130 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:36:21.932058 (+ 928us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:36:21.932061 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:36:21.932622 (+ 561us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:36:21.932630 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:36:21.933046 (+ 416us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:36:21.933049 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:36:21.933129 (+ 80us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:36:21.933682 (+ 553us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 15:36:21.933697 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0809 15:36:21.935638 (+ 1941us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.3.89.254:39259: Server oryx/127.3.89.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":246,"thread_start_us":106,"threads_started":1}
W20260809 15:36:21.935930 13340 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 15:36:21.929888 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.89.193:54467 (local address 127.3.89.254:39259)
0809 15:36:21.930172 (+ 284us) server_negotiation.cc:206] Beginning negotiation
0809 15:36:21.930175 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0809 15:36:21.930203 (+ 28us) server_negotiation.cc:407] Connection header received
0809 15:36:21.930410 (+ 207us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 15:36:21.930413 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 15:36:21.930489 (+ 76us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 15:36:21.930585 (+ 96us) server_negotiation.cc:226] Negotiated authn=SASL
0809 15:36:21.931260 (+ 675us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:36:21.931950 (+ 690us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:36:21.932723 (+ 773us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:36:21.932937 (+ 214us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:36:21.933013 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:36:21.935789 (+ 2776us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.3.89.193:54467: BlockingRecv error: recv got EOF from 127.3.89.193:54467 (error 108)
Metrics: {"server-negotiator.queue_time_us":215,"thread_start_us":108,"threads_started":1}
Aug 09 15:36:22 dist-test-slave-2w3w krb5kdc[12252](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.3.89.194@KRBTEST.COM for oryx/127.3.89.254@KRBTEST.COM, Server not found in Kerberos database
Aug 09 15:36:22 dist-test-slave-2w3w krb5kdc[12252](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.3.89.194@KRBTEST.COM for oryx/127.3.89.254@KRBTEST.COM, Server not found in Kerberos database
I20260809 15:36:22.119666 13341 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:36:22.113993 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.89.254:39259 (local address 127.3.89.194:42607)
0809 15:36:22.114304 (+ 311us) negotiation.cc:107] Waiting for socket to connect
0809 15:36:22.114318 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0809 15:36:22.114411 (+ 93us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 15:36:22.114640 (+ 229us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 15:36:22.114642 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 15:36:22.114859 (+ 217us) client_negotiation.cc:189] Negotiated authn=SASL
0809 15:36:22.115000 (+ 141us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:36:22.115006 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:36:22.115729 (+ 723us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:36:22.115732 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:36:22.116375 (+ 643us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:36:22.116384 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:36:22.116817 (+ 433us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:36:22.116819 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:36:22.116881 (+ 62us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:36:22.117500 (+ 619us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 15:36:22.117511 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0809 15:36:22.119555 (+ 2044us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.3.89.254:39259: Server oryx/127.3.89.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":228,"thread_start_us":111,"threads_started":1}
W20260809 15:36:22.119839 13340 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 15:36:22.114168 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.89.194:42607 (local address 127.3.89.254:39259)
0809 15:36:22.114330 (+ 162us) server_negotiation.cc:206] Beginning negotiation
0809 15:36:22.114333 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0809 15:36:22.114368 (+ 35us) server_negotiation.cc:407] Connection header received
0809 15:36:22.114494 (+ 126us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 15:36:22.114496 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 15:36:22.114537 (+ 41us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 15:36:22.114603 (+ 66us) server_negotiation.cc:226] Negotiated authn=SASL
0809 15:36:22.115109 (+ 506us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:36:22.115637 (+ 528us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:36:22.116496 (+ 859us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:36:22.116719 (+ 223us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:36:22.116821 (+ 102us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:36:22.119685 (+ 2864us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.3.89.194:42607: BlockingRecv error: recv got EOF from 127.3.89.194:42607 (error 108)
Metrics: {"server-negotiator.queue_time_us":66}
Aug 09 15:36:22 dist-test-slave-2w3w krb5kdc[12252](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.3.89.195@KRBTEST.COM for oryx/127.3.89.254@KRBTEST.COM, Server not found in Kerberos database
Aug 09 15:36:22 dist-test-slave-2w3w krb5kdc[12252](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.3.89.195@KRBTEST.COM for oryx/127.3.89.254@KRBTEST.COM, Server not found in Kerberos database
I20260809 15:36:22.304431 13342 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:36:22.298475 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.89.254:39259 (local address 127.3.89.195:46997)
0809 15:36:22.298816 (+ 341us) negotiation.cc:107] Waiting for socket to connect
0809 15:36:22.298830 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0809 15:36:22.298959 (+ 129us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 15:36:22.299259 (+ 300us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 15:36:22.299262 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 15:36:22.299495 (+ 233us) client_negotiation.cc:189] Negotiated authn=SASL
0809 15:36:22.299587 (+ 92us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:36:22.299593 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:36:22.300383 (+ 790us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:36:22.300389 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:36:22.300976 (+ 587us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:36:22.300985 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:36:22.301467 (+ 482us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:36:22.301471 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:36:22.301559 (+ 88us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:36:22.302120 (+ 561us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 15:36:22.302131 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0809 15:36:22.304319 (+ 2188us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.3.89.254:39259: Server oryx/127.3.89.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":255,"thread_start_us":86,"threads_started":1}
W20260809 15:36:22.304610 13340 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 15:36:22.298595 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.89.195:46997 (local address 127.3.89.254:39259)
0809 15:36:22.298756 (+ 161us) server_negotiation.cc:206] Beginning negotiation
0809 15:36:22.298760 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 15:36:22.298906 (+ 146us) server_negotiation.cc:407] Connection header received
0809 15:36:22.299089 (+ 183us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 15:36:22.299092 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 15:36:22.299143 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 15:36:22.299236 (+ 93us) server_negotiation.cc:226] Negotiated authn=SASL
0809 15:36:22.299713 (+ 477us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:36:22.300268 (+ 555us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:36:22.301102 (+ 834us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:36:22.301350 (+ 248us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:36:22.301464 (+ 114us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:36:22.304453 (+ 2989us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.3.89.195:46997: BlockingRecv error: recv got EOF from 127.3.89.195:46997 (error 108)
Metrics: {"server-negotiator.queue_time_us":65}
Aug 09 15:36:22 dist-test-slave-2w3w krb5kdc[12252](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.3.89.193@KRBTEST.COM for oryx/127.3.89.254@KRBTEST.COM, Server not found in Kerberos database
Aug 09 15:36:22 dist-test-slave-2w3w krb5kdc[12252](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.3.89.193@KRBTEST.COM for oryx/127.3.89.254@KRBTEST.COM, Server not found in Kerberos database
I20260809 15:36:22.942231 13343 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:36:22.936597 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.89.254:39259 (local address 127.3.89.193:36517)
0809 15:36:22.936883 (+ 286us) negotiation.cc:107] Waiting for socket to connect
0809 15:36:22.936897 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0809 15:36:22.937002 (+ 105us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 15:36:22.937239 (+ 237us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 15:36:22.937241 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 15:36:22.937432 (+ 191us) client_negotiation.cc:189] Negotiated authn=SASL
0809 15:36:22.937531 (+ 99us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:36:22.937537 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:36:22.938404 (+ 867us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:36:22.938407 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:36:22.939010 (+ 603us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:36:22.939019 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:36:22.939476 (+ 457us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:36:22.939478 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:36:22.939598 (+ 120us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:36:22.940165 (+ 567us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 15:36:22.940177 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0809 15:36:22.942117 (+ 1940us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.3.89.254:39259: Server oryx/127.3.89.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":207,"thread_start_us":96,"threads_started":1}
W20260809 15:36:22.942427 13344 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 15:36:22.936652 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.89.193:36517 (local address 127.3.89.254:39259)
0809 15:36:22.936905 (+ 253us) server_negotiation.cc:206] Beginning negotiation
0809 15:36:22.936908 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0809 15:36:22.936975 (+ 67us) server_negotiation.cc:407] Connection header received
0809 15:36:22.937080 (+ 105us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 15:36:22.937083 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 15:36:22.937126 (+ 43us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 15:36:22.937203 (+ 77us) server_negotiation.cc:226] Negotiated authn=SASL
0809 15:36:22.937665 (+ 462us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:36:22.938302 (+ 637us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:36:22.939121 (+ 819us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:36:22.939366 (+ 245us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:36:22.939475 (+ 109us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:36:22.942255 (+ 2780us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.3.89.193:36517: BlockingRecv error: recv got EOF from 127.3.89.193:36517 (error 108)
Metrics: {"server-negotiator.queue_time_us":171,"thread_start_us":62,"threads_started":1}
Aug 09 15:36:23 dist-test-slave-2w3w krb5kdc[12252](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.3.89.194@KRBTEST.COM for oryx/127.3.89.254@KRBTEST.COM, Server not found in Kerberos database
Aug 09 15:36:23 dist-test-slave-2w3w krb5kdc[12252](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.3.89.194@KRBTEST.COM for oryx/127.3.89.254@KRBTEST.COM, Server not found in Kerberos database
I20260809 15:36:23.125691 13345 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:36:23.120427 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.89.254:39259 (local address 127.3.89.194:41813)
0809 15:36:23.120754 (+ 327us) negotiation.cc:107] Waiting for socket to connect
0809 15:36:23.120768 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0809 15:36:23.120869 (+ 101us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 15:36:23.121111 (+ 242us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 15:36:23.121113 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 15:36:23.121299 (+ 186us) client_negotiation.cc:189] Negotiated authn=SASL
0809 15:36:23.121398 (+ 99us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:36:23.121403 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:36:23.122183 (+ 780us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:36:23.122186 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:36:23.122730 (+ 544us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:36:23.122737 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:36:23.123228 (+ 491us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:36:23.123231 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:36:23.123294 (+ 63us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:36:23.123752 (+ 458us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 15:36:23.123762 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0809 15:36:23.125577 (+ 1815us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.3.89.254:39259: Server oryx/127.3.89.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":252,"thread_start_us":87,"threads_started":1}
W20260809 15:36:23.125905 13344 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 15:36:23.120542 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.89.194:41813 (local address 127.3.89.254:39259)
0809 15:36:23.120694 (+ 152us) server_negotiation.cc:206] Beginning negotiation
0809 15:36:23.120698 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 15:36:23.120843 (+ 145us) server_negotiation.cc:407] Connection header received
0809 15:36:23.120952 (+ 109us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 15:36:23.120954 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 15:36:23.121000 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 15:36:23.121078 (+ 78us) server_negotiation.cc:226] Negotiated authn=SASL
0809 15:36:23.121533 (+ 455us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:36:23.122082 (+ 549us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:36:23.122834 (+ 752us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:36:23.123130 (+ 296us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:36:23.123211 (+ 81us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:36:23.125764 (+ 2553us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.3.89.194:41813: BlockingRecv error: recv got EOF from 127.3.89.194:41813 (error 108)
Metrics: {"server-negotiator.queue_time_us":52}
Aug 09 15:36:23 dist-test-slave-2w3w krb5kdc[12252](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.3.89.195@KRBTEST.COM for oryx/127.3.89.254@KRBTEST.COM, Server not found in Kerberos database
Aug 09 15:36:23 dist-test-slave-2w3w krb5kdc[12252](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.3.89.195@KRBTEST.COM for oryx/127.3.89.254@KRBTEST.COM, Server not found in Kerberos database
I20260809 15:36:23.311172 13346 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:36:23.305132 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.89.254:39259 (local address 127.3.89.195:52019)
0809 15:36:23.305502 (+ 370us) negotiation.cc:107] Waiting for socket to connect
0809 15:36:23.305524 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0809 15:36:23.305655 (+ 131us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 15:36:23.305962 (+ 307us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 15:36:23.305965 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 15:36:23.306184 (+ 219us) client_negotiation.cc:189] Negotiated authn=SASL
0809 15:36:23.306276 (+ 92us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:36:23.306289 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:36:23.307138 (+ 849us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:36:23.307143 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:36:23.307736 (+ 593us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:36:23.307745 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:36:23.308261 (+ 516us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:36:23.308265 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:36:23.308360 (+ 95us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:36:23.308926 (+ 566us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 15:36:23.308938 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0809 15:36:23.310988 (+ 2050us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.3.89.254:39259: Server oryx/127.3.89.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":287,"thread_start_us":110,"threads_started":1}
W20260809 15:36:23.311257 13344 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 15:36:23.305275 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.89.195:52019 (local address 127.3.89.254:39259)
0809 15:36:23.305455 (+ 180us) server_negotiation.cc:206] Beginning negotiation
0809 15:36:23.305460 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 15:36:23.305602 (+ 142us) server_negotiation.cc:407] Connection header received
0809 15:36:23.305761 (+ 159us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 15:36:23.305765 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 15:36:23.305826 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 15:36:23.305920 (+ 94us) server_negotiation.cc:226] Negotiated authn=SASL
0809 15:36:23.306407 (+ 487us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:36:23.307023 (+ 616us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:36:23.307897 (+ 874us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:36:23.308143 (+ 246us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:36:23.308256 (+ 113us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:36:23.311143 (+ 2887us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.3.89.195:52019: BlockingRecv error: recv got EOF from 127.3.89.195:52019 (error 108)
Metrics: {"server-negotiator.queue_time_us":77}
Aug 09 15:36:23 dist-test-slave-2w3w krb5kdc[12252](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.3.89.193@KRBTEST.COM for oryx/127.3.89.254@KRBTEST.COM, Server not found in Kerberos database
Aug 09 15:36:23 dist-test-slave-2w3w krb5kdc[12252](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.3.89.193@KRBTEST.COM for oryx/127.3.89.254@KRBTEST.COM, Server not found in Kerberos database
I20260809 15:36:23.948927 13348 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:36:23.943015 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.89.254:39259 (local address 127.3.89.193:33505)
0809 15:36:23.943338 (+ 323us) negotiation.cc:107] Waiting for socket to connect
0809 15:36:23.943355 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0809 15:36:23.943480 (+ 125us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 15:36:23.943790 (+ 310us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 15:36:23.943792 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 15:36:23.944008 (+ 216us) client_negotiation.cc:189] Negotiated authn=SASL
0809 15:36:23.944107 (+ 99us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:36:23.944112 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:36:23.944891 (+ 779us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:36:23.944895 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:36:23.945442 (+ 547us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:36:23.945450 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:36:23.945913 (+ 463us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:36:23.945916 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:36:23.946003 (+ 87us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:36:23.946625 (+ 622us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 15:36:23.946637 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0809 15:36:23.948711 (+ 2074us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.3.89.254:39259: Server oryx/127.3.89.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":236,"thread_start_us":92,"threads_started":1}
W20260809 15:36:23.948989 13347 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 15:36:23.943046 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.89.193:33505 (local address 127.3.89.254:39259)
0809 15:36:23.943443 (+ 397us) server_negotiation.cc:206] Beginning negotiation
0809 15:36:23.943447 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 15:36:23.943461 (+ 14us) server_negotiation.cc:407] Connection header received
0809 15:36:23.943609 (+ 148us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 15:36:23.943613 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 15:36:23.943666 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 15:36:23.943764 (+ 98us) server_negotiation.cc:226] Negotiated authn=SASL
0809 15:36:23.944243 (+ 479us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:36:23.944776 (+ 533us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:36:23.945565 (+ 789us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:36:23.945799 (+ 234us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:36:23.945907 (+ 108us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:36:23.948865 (+ 2958us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.3.89.193:33505: BlockingRecv error: recv got EOF from 127.3.89.193:33505 (error 108)
Metrics: {"server-negotiator.queue_time_us":300,"thread_start_us":84,"threads_started":1}
Aug 09 15:36:24 dist-test-slave-2w3w krb5kdc[12252](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.3.89.194@KRBTEST.COM for oryx/127.3.89.254@KRBTEST.COM, Server not found in Kerberos database
Aug 09 15:36:24 dist-test-slave-2w3w krb5kdc[12252](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.3.89.194@KRBTEST.COM for oryx/127.3.89.254@KRBTEST.COM, Server not found in Kerberos database
I20260809 15:36:24.131999 13349 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:36:24.126440 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.89.254:39259 (local address 127.3.89.194:33773)
0809 15:36:24.126820 (+ 380us) negotiation.cc:107] Waiting for socket to connect
0809 15:36:24.126833 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0809 15:36:24.126936 (+ 103us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 15:36:24.127188 (+ 252us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 15:36:24.127190 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 15:36:24.127374 (+ 184us) client_negotiation.cc:189] Negotiated authn=SASL
0809 15:36:24.127480 (+ 106us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:36:24.127486 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:36:24.128266 (+ 780us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:36:24.128269 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:36:24.128857 (+ 588us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:36:24.128866 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:36:24.129385 (+ 519us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:36:24.129388 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:36:24.129451 (+ 63us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:36:24.129955 (+ 504us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 15:36:24.129974 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0809 15:36:24.131886 (+ 1912us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.3.89.254:39259: Server oryx/127.3.89.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":292,"thread_start_us":88,"threads_started":1}
W20260809 15:36:24.132138 13347 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 15:36:24.126577 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.89.194:33773 (local address 127.3.89.254:39259)
0809 15:36:24.126733 (+ 156us) server_negotiation.cc:206] Beginning negotiation
0809 15:36:24.126738 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 15:36:24.126905 (+ 167us) server_negotiation.cc:407] Connection header received
0809 15:36:24.127032 (+ 127us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 15:36:24.127034 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 15:36:24.127078 (+ 44us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 15:36:24.127150 (+ 72us) server_negotiation.cc:226] Negotiated authn=SASL
0809 15:36:24.127578 (+ 428us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:36:24.128173 (+ 595us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:36:24.128987 (+ 814us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:36:24.129284 (+ 297us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:36:24.129367 (+ 83us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:36:24.132016 (+ 2649us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.3.89.194:33773: BlockingRecv error: recv got EOF from 127.3.89.194:33773 (error 108)
Metrics: {"server-negotiator.queue_time_us":58}
Aug 09 15:36:24 dist-test-slave-2w3w krb5kdc[12252](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.3.89.195@KRBTEST.COM for oryx/127.3.89.254@KRBTEST.COM, Server not found in Kerberos database
Aug 09 15:36:24 dist-test-slave-2w3w krb5kdc[12252](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.3.89.195@KRBTEST.COM for oryx/127.3.89.254@KRBTEST.COM, Server not found in Kerberos database
I20260809 15:36:24.317625 13350 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:36:24.311910 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.89.254:39259 (local address 127.3.89.195:56261)
0809 15:36:24.312221 (+ 311us) negotiation.cc:107] Waiting for socket to connect
0809 15:36:24.312234 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0809 15:36:24.312334 (+ 100us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 15:36:24.312566 (+ 232us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 15:36:24.312568 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 15:36:24.312800 (+ 232us) client_negotiation.cc:189] Negotiated authn=SASL
0809 15:36:24.312894 (+ 94us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:36:24.312899 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:36:24.313667 (+ 768us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:36:24.313670 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:36:24.314235 (+ 565us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:36:24.314243 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:36:24.314670 (+ 427us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:36:24.314672 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:36:24.314745 (+ 73us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:36:24.315325 (+ 580us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 15:36:24.315337 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0809 15:36:24.317413 (+ 2076us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.3.89.254:39259: Server oryx/127.3.89.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":241,"thread_start_us":112,"threads_started":1}
W20260809 15:36:24.317762 13347 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 15:36:24.311998 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.89.195:56261 (local address 127.3.89.254:39259)
0809 15:36:24.312195 (+ 197us) server_negotiation.cc:206] Beginning negotiation
0809 15:36:24.312200 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 15:36:24.312333 (+ 133us) server_negotiation.cc:407] Connection header received
0809 15:36:24.312415 (+ 82us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 15:36:24.312417 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 15:36:24.312465 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 15:36:24.312533 (+ 68us) server_negotiation.cc:226] Negotiated authn=SASL
0809 15:36:24.313035 (+ 502us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:36:24.313569 (+ 534us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:36:24.314342 (+ 773us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:36:24.314571 (+ 229us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:36:24.314651 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:36:24.317638 (+ 2987us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.3.89.195:56261: BlockingRecv error: recv got EOF from 127.3.89.195:56261 (error 108)
Metrics: {"server-negotiator.queue_time_us":89}
Aug 09 15:36:24 dist-test-slave-2w3w krb5kdc[12252](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.3.89.193@KRBTEST.COM for oryx/127.3.89.254@KRBTEST.COM, Server not found in Kerberos database
Aug 09 15:36:24 dist-test-slave-2w3w krb5kdc[12252](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.3.89.193@KRBTEST.COM for oryx/127.3.89.254@KRBTEST.COM, Server not found in Kerberos database
I20260809 15:36:24.955395 13351 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:36:24.949875 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.89.254:39259 (local address 127.3.89.193:35223)
0809 15:36:24.950249 (+ 374us) negotiation.cc:107] Waiting for socket to connect
0809 15:36:24.950266 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0809 15:36:24.950374 (+ 108us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 15:36:24.950659 (+ 285us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 15:36:24.950661 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 15:36:24.950909 (+ 248us) client_negotiation.cc:189] Negotiated authn=SASL
0809 15:36:24.951086 (+ 177us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:36:24.951092 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:36:24.951844 (+ 752us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:36:24.951846 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:36:24.952383 (+ 537us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:36:24.952390 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:36:24.952787 (+ 397us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:36:24.952789 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:36:24.952850 (+ 61us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:36:24.953370 (+ 520us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 15:36:24.953381 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0809 15:36:24.955283 (+ 1902us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.3.89.254:39259: Server oryx/127.3.89.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":273,"thread_start_us":101,"threads_started":1}
W20260809 15:36:24.955595 13352 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 15:36:24.949984 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.89.193:35223 (local address 127.3.89.254:39259)
0809 15:36:24.950245 (+ 261us) server_negotiation.cc:206] Beginning negotiation
0809 15:36:24.950248 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0809 15:36:24.950330 (+ 82us) server_negotiation.cc:407] Connection header received
0809 15:36:24.950463 (+ 133us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 15:36:24.950467 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 15:36:24.950520 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 15:36:24.950621 (+ 101us) server_negotiation.cc:226] Negotiated authn=SASL
0809 15:36:24.951188 (+ 567us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:36:24.951703 (+ 515us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:36:24.952487 (+ 784us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:36:24.952697 (+ 210us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:36:24.952795 (+ 98us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:36:24.955422 (+ 2627us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.3.89.193:35223: BlockingRecv error: recv got EOF from 127.3.89.193:35223 (error 108)
Metrics: {"server-negotiator.queue_time_us":161,"thread_start_us":83,"threads_started":1}
Aug 09 15:36:25 dist-test-slave-2w3w krb5kdc[12252](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.3.89.194@KRBTEST.COM for oryx/127.3.89.254@KRBTEST.COM, Server not found in Kerberos database
Aug 09 15:36:25 dist-test-slave-2w3w krb5kdc[12252](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.3.89.194@KRBTEST.COM for oryx/127.3.89.254@KRBTEST.COM, Server not found in Kerberos database
I20260809 15:36:25.139003 13353 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:36:25.132880 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.89.254:39259 (local address 127.3.89.194:34207)
0809 15:36:25.133170 (+ 290us) negotiation.cc:107] Waiting for socket to connect
0809 15:36:25.133185 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0809 15:36:25.133324 (+ 139us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 15:36:25.133609 (+ 285us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 15:36:25.133612 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 15:36:25.133828 (+ 216us) client_negotiation.cc:189] Negotiated authn=SASL
0809 15:36:25.133921 (+ 93us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:36:25.133926 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:36:25.134886 (+ 960us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:36:25.134889 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:36:25.135682 (+ 793us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:36:25.135696 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:36:25.136288 (+ 592us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:36:25.136291 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:36:25.136355 (+ 64us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:36:25.136930 (+ 575us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 15:36:25.136942 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0809 15:36:25.138834 (+ 1892us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.3.89.254:39259: Server oryx/127.3.89.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":212,"thread_start_us":88,"threads_started":1}
W20260809 15:36:25.139148 13352 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 15:36:25.132963 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.89.194:34207 (local address 127.3.89.254:39259)
0809 15:36:25.133138 (+ 175us) server_negotiation.cc:206] Beginning negotiation
0809 15:36:25.133142 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 15:36:25.133270 (+ 128us) server_negotiation.cc:407] Connection header received
0809 15:36:25.133439 (+ 169us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 15:36:25.133442 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 15:36:25.133497 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 15:36:25.133587 (+ 90us) server_negotiation.cc:226] Negotiated authn=SASL
0809 15:36:25.134044 (+ 457us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:36:25.134713 (+ 669us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:36:25.135797 (+ 1084us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:36:25.136155 (+ 358us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:36:25.136251 (+ 96us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:36:25.139011 (+ 2760us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.3.89.194:34207: BlockingRecv error: recv got EOF from 127.3.89.194:34207 (error 108)
Metrics: {"server-negotiator.queue_time_us":73}
Aug 09 15:36:25 dist-test-slave-2w3w krb5kdc[12252](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.3.89.195@KRBTEST.COM for oryx/127.3.89.254@KRBTEST.COM, Server not found in Kerberos database
Aug 09 15:36:25 dist-test-slave-2w3w krb5kdc[12252](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.3.89.195@KRBTEST.COM for oryx/127.3.89.254@KRBTEST.COM, Server not found in Kerberos database
I20260809 15:36:25.323952 13354 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:36:25.318467 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.89.254:39259 (local address 127.3.89.195:59681)
0809 15:36:25.318790 (+ 323us) negotiation.cc:107] Waiting for socket to connect
0809 15:36:25.318813 (+ 23us) client_negotiation.cc:174] Beginning negotiation
0809 15:36:25.318917 (+ 104us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 15:36:25.319269 (+ 352us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 15:36:25.319273 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 15:36:25.319459 (+ 186us) client_negotiation.cc:189] Negotiated authn=SASL
0809 15:36:25.319557 (+ 98us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:36:25.319562 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:36:25.320335 (+ 773us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:36:25.320339 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:36:25.320887 (+ 548us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:36:25.320896 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:36:25.321336 (+ 440us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:36:25.321339 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:36:25.321403 (+ 64us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:36:25.321896 (+ 493us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 15:36:25.321907 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0809 15:36:25.323841 (+ 1934us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.3.89.254:39259: Server oryx/127.3.89.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":180,"thread_start_us":91,"threads_started":1}
W20260809 15:36:25.324091 13352 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 15:36:25.318551 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.89.195:59681 (local address 127.3.89.254:39259)
0809 15:36:25.318687 (+ 136us) server_negotiation.cc:206] Beginning negotiation
0809 15:36:25.318691 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 15:36:25.318891 (+ 200us) server_negotiation.cc:407] Connection header received
0809 15:36:25.319067 (+ 176us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 15:36:25.319069 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 15:36:25.319116 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 15:36:25.319193 (+ 77us) server_negotiation.cc:226] Negotiated authn=SASL
0809 15:36:25.319675 (+ 482us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:36:25.320210 (+ 535us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:36:25.320997 (+ 787us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:36:25.321210 (+ 213us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:36:25.321294 (+ 84us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:36:25.323975 (+ 2681us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.3.89.195:59681: BlockingRecv error: recv got EOF from 127.3.89.195:59681 (error 108)
Metrics: {"server-negotiator.queue_time_us":49}
Aug 09 15:36:25 dist-test-slave-2w3w krb5kdc[12252](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.3.89.193@KRBTEST.COM for oryx/127.3.89.254@KRBTEST.COM, Server not found in Kerberos database
Aug 09 15:36:25 dist-test-slave-2w3w krb5kdc[12252](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.3.89.193@KRBTEST.COM for oryx/127.3.89.254@KRBTEST.COM, Server not found in Kerberos database
I20260809 15:36:25.965922 13355 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:36:25.956143 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.89.254:39259 (local address 127.3.89.193:52927)
0809 15:36:25.956448 (+ 305us) negotiation.cc:107] Waiting for socket to connect
0809 15:36:25.956462 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0809 15:36:25.956553 (+ 91us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 15:36:25.956798 (+ 245us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 15:36:25.956800 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 15:36:25.957004 (+ 204us) client_negotiation.cc:189] Negotiated authn=SASL
0809 15:36:25.957100 (+ 96us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:36:25.957105 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:36:25.957865 (+ 760us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:36:25.957869 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:36:25.958624 (+ 755us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:36:25.958634 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:36:25.959211 (+ 577us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:36:25.959217 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:36:25.959295 (+ 78us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:36:25.963796 (+ 4501us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 15:36:25.963808 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0809 15:36:25.965783 (+ 1975us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.3.89.254:39259: Server oryx/127.3.89.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":207,"thread_start_us":90,"threads_started":1}
W20260809 15:36:25.966084 13357 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 15:36:25.956266 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.89.193:52927 (local address 127.3.89.254:39259)
0809 15:36:25.956514 (+ 248us) server_negotiation.cc:206] Beginning negotiation
0809 15:36:25.956518 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 15:36:25.956531 (+ 13us) server_negotiation.cc:407] Connection header received
0809 15:36:25.956642 (+ 111us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 15:36:25.956644 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 15:36:25.956685 (+ 41us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 15:36:25.956760 (+ 75us) server_negotiation.cc:226] Negotiated authn=SASL
0809 15:36:25.957199 (+ 439us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:36:25.957731 (+ 532us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:36:25.958756 (+ 1025us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:36:25.959076 (+ 320us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:36:25.959187 (+ 111us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:36:25.965935 (+ 6748us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.3.89.193:52927: BlockingRecv error: recv got EOF from 127.3.89.193:52927 (error 108)
Metrics: {"server-negotiator.queue_time_us":168,"thread_start_us":61,"threads_started":1}
Aug 09 15:36:26 dist-test-slave-2w3w krb5kdc[12252](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.3.89.194@KRBTEST.COM for oryx/127.3.89.254@KRBTEST.COM, Server not found in Kerberos database
Aug 09 15:36:26 dist-test-slave-2w3w krb5kdc[12252](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.3.89.194@KRBTEST.COM for oryx/127.3.89.254@KRBTEST.COM, Server not found in Kerberos database
I20260809 15:36:26.145370 13358 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:36:26.139730 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.89.254:39259 (local address 127.3.89.194:52007)
0809 15:36:26.140030 (+ 300us) negotiation.cc:107] Waiting for socket to connect
0809 15:36:26.140043 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0809 15:36:26.140185 (+ 142us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 15:36:26.140481 (+ 296us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 15:36:26.140484 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 15:36:26.140732 (+ 248us) client_negotiation.cc:189] Negotiated authn=SASL
0809 15:36:26.140840 (+ 108us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:36:26.140846 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:36:26.141728 (+ 882us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:36:26.141733 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:36:26.142301 (+ 568us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:36:26.142309 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:36:26.142770 (+ 461us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:36:26.142773 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:36:26.142862 (+ 89us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:36:26.143359 (+ 497us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 15:36:26.143371 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0809 15:36:26.145238 (+ 1867us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.3.89.254:39259: Server oryx/127.3.89.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":217,"thread_start_us":83,"threads_started":1}
W20260809 15:36:26.145481 13357 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 15:36:26.139792 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.89.194:52007 (local address 127.3.89.254:39259)
0809 15:36:26.139967 (+ 175us) server_negotiation.cc:206] Beginning negotiation
0809 15:36:26.139973 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 15:36:26.140122 (+ 149us) server_negotiation.cc:407] Connection header received
0809 15:36:26.140291 (+ 169us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 15:36:26.140295 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 15:36:26.140347 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 15:36:26.140435 (+ 88us) server_negotiation.cc:226] Negotiated authn=SASL
0809 15:36:26.140978 (+ 543us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:36:26.141612 (+ 634us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:36:26.142425 (+ 813us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:36:26.142654 (+ 229us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:36:26.142767 (+ 113us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:36:26.145365 (+ 2598us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.3.89.194:52007: BlockingRecv error: recv got EOF from 127.3.89.194:52007 (error 108)
Metrics: {"server-negotiator.queue_time_us":67}
Aug 09 15:36:26 dist-test-slave-2w3w krb5kdc[12252](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.3.89.195@KRBTEST.COM for oryx/127.3.89.254@KRBTEST.COM, Server not found in Kerberos database
Aug 09 15:36:26 dist-test-slave-2w3w krb5kdc[12252](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.3.89.195@KRBTEST.COM for oryx/127.3.89.254@KRBTEST.COM, Server not found in Kerberos database
I20260809 15:36:26.330417 13359 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:36:26.324797 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.89.254:39259 (local address 127.3.89.195:52579)
0809 15:36:26.325103 (+ 306us) negotiation.cc:107] Waiting for socket to connect
0809 15:36:26.325117 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0809 15:36:26.325214 (+ 97us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 15:36:26.325449 (+ 235us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 15:36:26.325452 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 15:36:26.325638 (+ 186us) client_negotiation.cc:189] Negotiated authn=SASL
0809 15:36:26.325731 (+ 93us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:36:26.325737 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:36:26.326472 (+ 735us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:36:26.326474 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:36:26.327073 (+ 599us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:36:26.327081 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:36:26.327496 (+ 415us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:36:26.327499 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:36:26.327560 (+ 61us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:36:26.328113 (+ 553us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 15:36:26.328124 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0809 15:36:26.330252 (+ 2128us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.3.89.254:39259: Server oryx/127.3.89.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":233,"thread_start_us":104,"threads_started":1}
W20260809 15:36:26.330475 13357 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 15:36:26.324873 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.89.195:52579 (local address 127.3.89.254:39259)
0809 15:36:26.325014 (+ 141us) server_negotiation.cc:206] Beginning negotiation
0809 15:36:26.325019 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 15:36:26.325219 (+ 200us) server_negotiation.cc:407] Connection header received
0809 15:36:26.325288 (+ 69us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 15:36:26.325290 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 15:36:26.325343 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 15:36:26.325414 (+ 71us) server_negotiation.cc:226] Negotiated authn=SASL
0809 15:36:26.325863 (+ 449us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:36:26.326369 (+ 506us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:36:26.327169 (+ 800us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:36:26.327389 (+ 220us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:36:26.327469 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:36:26.330369 (+ 2900us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.3.89.195:52579: BlockingRecv error: recv got EOF from 127.3.89.195:52579 (error 108)
Metrics: {"server-negotiator.queue_time_us":54}
Aug 09 15:36:26 dist-test-slave-2w3w krb5kdc[12252](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.3.89.193@KRBTEST.COM for oryx/127.3.89.254@KRBTEST.COM, Server not found in Kerberos database
Aug 09 15:36:26 dist-test-slave-2w3w krb5kdc[12252](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.3.89.193@KRBTEST.COM for oryx/127.3.89.254@KRBTEST.COM, Server not found in Kerberos database
I20260809 15:36:26.972306 13360 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:36:26.966910 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.89.254:39259 (local address 127.3.89.193:51173)
0809 15:36:26.967244 (+ 334us) negotiation.cc:107] Waiting for socket to connect
0809 15:36:26.967258 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0809 15:36:26.967342 (+ 84us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 15:36:26.967563 (+ 221us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 15:36:26.967565 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 15:36:26.967748 (+ 183us) client_negotiation.cc:189] Negotiated authn=SASL
0809 15:36:26.967841 (+ 93us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:36:26.967846 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:36:26.968633 (+ 787us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:36:26.968635 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:36:26.969203 (+ 568us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:36:26.969210 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:36:26.969625 (+ 415us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:36:26.969628 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:36:26.969688 (+ 60us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:36:26.970251 (+ 563us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 15:36:26.970263 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0809 15:36:26.972190 (+ 1927us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.3.89.254:39259: Server oryx/127.3.89.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":239,"thread_start_us":121,"threads_started":1}
W20260809 15:36:26.972461 13361 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 15:36:26.967016 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.89.193:51173 (local address 127.3.89.254:39259)
0809 15:36:26.967320 (+ 304us) server_negotiation.cc:206] Beginning negotiation
0809 15:36:26.967323 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0809 15:36:26.967336 (+ 13us) server_negotiation.cc:407] Connection header received
0809 15:36:26.967412 (+ 76us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 15:36:26.967414 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 15:36:26.967455 (+ 41us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 15:36:26.967531 (+ 76us) server_negotiation.cc:226] Negotiated authn=SASL
0809 15:36:26.967983 (+ 452us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:36:26.968533 (+ 550us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:36:26.969316 (+ 783us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:36:26.969530 (+ 214us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:36:26.969605 (+ 75us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:36:26.972328 (+ 2723us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.3.89.193:51173: BlockingRecv error: recv got EOF from 127.3.89.193:51173 (error 108)
Metrics: {"server-negotiator.queue_time_us":233,"thread_start_us":115,"threads_started":1}
Aug 09 15:36:27 dist-test-slave-2w3w krb5kdc[12252](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.3.89.194@KRBTEST.COM for oryx/127.3.89.254@KRBTEST.COM, Server not found in Kerberos database
Aug 09 15:36:27 dist-test-slave-2w3w krb5kdc[12252](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.3.89.194@KRBTEST.COM for oryx/127.3.89.254@KRBTEST.COM, Server not found in Kerberos database
I20260809 15:36:27.151947 13362 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:36:27.146304 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.89.254:39259 (local address 127.3.89.194:41723)
0809 15:36:27.146585 (+ 281us) negotiation.cc:107] Waiting for socket to connect
0809 15:36:27.146598 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0809 15:36:27.146696 (+ 98us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 15:36:27.147024 (+ 328us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 15:36:27.147028 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 15:36:27.147269 (+ 241us) client_negotiation.cc:189] Negotiated authn=SASL
0809 15:36:27.147401 (+ 132us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:36:27.147427 (+ 26us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:36:27.148466 (+ 1039us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:36:27.148469 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:36:27.149026 (+ 557us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:36:27.149033 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:36:27.149437 (+ 404us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:36:27.149440 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:36:27.149504 (+ 64us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:36:27.149987 (+ 483us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 15:36:27.149998 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0809 15:36:27.151836 (+ 1838us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.3.89.254:39259: Server oryx/127.3.89.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":207,"thread_start_us":97,"threads_started":1}
W20260809 15:36:27.152097 13361 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 15:36:27.146363 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.89.194:41723 (local address 127.3.89.254:39259)
0809 15:36:27.146521 (+ 158us) server_negotiation.cc:206] Beginning negotiation
0809 15:36:27.146528 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 15:36:27.146676 (+ 148us) server_negotiation.cc:407] Connection header received
0809 15:36:27.146802 (+ 126us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 15:36:27.146805 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 15:36:27.146860 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 15:36:27.146995 (+ 135us) server_negotiation.cc:226] Negotiated authn=SASL
0809 15:36:27.147564 (+ 569us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:36:27.148364 (+ 800us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:36:27.149125 (+ 761us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:36:27.149346 (+ 221us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:36:27.149419 (+ 73us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:36:27.151981 (+ 2562us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.3.89.194:41723: BlockingRecv error: recv got EOF from 127.3.89.194:41723 (error 108)
Metrics: {"server-negotiator.queue_time_us":60}
Aug 09 15:36:27 dist-test-slave-2w3w krb5kdc[12252](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.3.89.195@KRBTEST.COM for oryx/127.3.89.254@KRBTEST.COM, Server not found in Kerberos database
Aug 09 15:36:27 dist-test-slave-2w3w krb5kdc[12252](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.3.89.195@KRBTEST.COM for oryx/127.3.89.254@KRBTEST.COM, Server not found in Kerberos database
I20260809 15:36:27.336479 13363 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:36:27.331234 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.89.254:39259 (local address 127.3.89.195:48639)
0809 15:36:27.331495 (+ 261us) negotiation.cc:107] Waiting for socket to connect
0809 15:36:27.331508 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0809 15:36:27.331596 (+ 88us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 15:36:27.331796 (+ 200us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 15:36:27.331798 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 15:36:27.331971 (+ 173us) client_negotiation.cc:189] Negotiated authn=SASL
0809 15:36:27.332065 (+ 94us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:36:27.332070 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:36:27.332808 (+ 738us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:36:27.332810 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:36:27.333368 (+ 558us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:36:27.333375 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:36:27.333874 (+ 499us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:36:27.333877 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:36:27.333933 (+ 56us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:36:27.334470 (+ 537us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 15:36:27.334481 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0809 15:36:27.336365 (+ 1884us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.3.89.254:39259: Server oryx/127.3.89.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":182,"thread_start_us":95,"threads_started":1}
W20260809 15:36:27.336607 13361 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 15:36:27.331348 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.89.195:48639 (local address 127.3.89.254:39259)
0809 15:36:27.331492 (+ 144us) server_negotiation.cc:206] Beginning negotiation
0809 15:36:27.331496 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 15:36:27.331571 (+ 75us) server_negotiation.cc:407] Connection header received
0809 15:36:27.331671 (+ 100us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 15:36:27.331673 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 15:36:27.331718 (+ 45us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 15:36:27.331783 (+ 65us) server_negotiation.cc:226] Negotiated authn=SASL
0809 15:36:27.332156 (+ 373us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:36:27.332707 (+ 551us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:36:27.333467 (+ 760us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:36:27.333777 (+ 310us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:36:27.333855 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:36:27.336494 (+ 2639us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.3.89.195:48639: BlockingRecv error: recv got EOF from 127.3.89.195:48639 (error 108)
Metrics: {"server-negotiator.queue_time_us":55}
Aug 09 15:36:27 dist-test-slave-2w3w krb5kdc[12252](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.3.89.193@KRBTEST.COM for oryx/127.3.89.254@KRBTEST.COM, Server not found in Kerberos database
Aug 09 15:36:27 dist-test-slave-2w3w krb5kdc[12252](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.3.89.193@KRBTEST.COM for oryx/127.3.89.254@KRBTEST.COM, Server not found in Kerberos database
I20260809 15:36:27.978528 13364 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:36:27.973179 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.89.254:39259 (local address 127.3.89.193:40717)
0809 15:36:27.973488 (+ 309us) negotiation.cc:107] Waiting for socket to connect
0809 15:36:27.973501 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0809 15:36:27.973598 (+ 97us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 15:36:27.973854 (+ 256us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 15:36:27.973856 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 15:36:27.974038 (+ 182us) client_negotiation.cc:189] Negotiated authn=SASL
0809 15:36:27.974136 (+ 98us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:36:27.974141 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:36:27.974888 (+ 747us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:36:27.974891 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:36:27.975497 (+ 606us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:36:27.975506 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:36:27.975916 (+ 410us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:36:27.975918 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:36:27.975982 (+ 64us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:36:27.976550 (+ 568us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 15:36:27.976562 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0809 15:36:27.978406 (+ 1844us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.3.89.254:39259: Server oryx/127.3.89.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":220,"thread_start_us":119,"threads_started":1}
W20260809 15:36:27.978684 13365 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 15:36:27.973230 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.89.193:40717 (local address 127.3.89.254:39259)
0809 15:36:27.973474 (+ 244us) server_negotiation.cc:206] Beginning negotiation
0809 15:36:27.973477 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0809 15:36:27.973570 (+ 93us) server_negotiation.cc:407] Connection header received
0809 15:36:27.973678 (+ 108us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 15:36:27.973680 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 15:36:27.973735 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 15:36:27.973813 (+ 78us) server_negotiation.cc:226] Negotiated authn=SASL
0809 15:36:27.974230 (+ 417us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:36:27.974753 (+ 523us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:36:27.975597 (+ 844us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:36:27.975811 (+ 214us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:36:27.975888 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:36:27.978545 (+ 2657us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.3.89.193:40717: BlockingRecv error: recv got EOF from 127.3.89.193:40717 (error 108)
Metrics: {"server-negotiator.queue_time_us":183,"thread_start_us":87,"threads_started":1}
Aug 09 15:36:28 dist-test-slave-2w3w krb5kdc[12252](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.3.89.194@KRBTEST.COM for oryx/127.3.89.254@KRBTEST.COM, Server not found in Kerberos database
Aug 09 15:36:28 dist-test-slave-2w3w krb5kdc[12252](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.3.89.194@KRBTEST.COM for oryx/127.3.89.254@KRBTEST.COM, Server not found in Kerberos database
I20260809 15:36:28.158504 13366 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:36:28.152747 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.89.254:39259 (local address 127.3.89.194:39275)
0809 15:36:28.153045 (+ 298us) negotiation.cc:107] Waiting for socket to connect
0809 15:36:28.153062 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0809 15:36:28.153247 (+ 185us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 15:36:28.153640 (+ 393us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 15:36:28.153644 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 15:36:28.153895 (+ 251us) client_negotiation.cc:189] Negotiated authn=SASL
0809 15:36:28.153991 (+ 96us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:36:28.154020 (+ 29us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:36:28.154855 (+ 835us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:36:28.154858 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:36:28.155439 (+ 581us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:36:28.155447 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:36:28.155850 (+ 403us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:36:28.155852 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:36:28.155918 (+ 66us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:36:28.156476 (+ 558us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 15:36:28.156490 (+ 14us) client_negotiation.cc:656] calling sasl_client_start()
0809 15:36:28.158393 (+ 1903us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.3.89.254:39259: Server oryx/127.3.89.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":222,"thread_start_us":105,"threads_started":1}
W20260809 15:36:28.158649 13365 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 15:36:28.152867 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.89.194:39275 (local address 127.3.89.254:39259)
0809 15:36:28.153032 (+ 165us) server_negotiation.cc:206] Beginning negotiation
0809 15:36:28.153037 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 15:36:28.153190 (+ 153us) server_negotiation.cc:407] Connection header received
0809 15:36:28.153348 (+ 158us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 15:36:28.153351 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 15:36:28.153430 (+ 79us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 15:36:28.153511 (+ 81us) server_negotiation.cc:226] Negotiated authn=SASL
0809 15:36:28.154129 (+ 618us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:36:28.154739 (+ 610us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:36:28.155538 (+ 799us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:36:28.155753 (+ 215us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:36:28.155828 (+ 75us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:36:28.158529 (+ 2701us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.3.89.194:39275: BlockingRecv error: recv got EOF from 127.3.89.194:39275 (error 108)
Metrics: {"server-negotiator.queue_time_us":75}
Aug 09 15:36:28 dist-test-slave-2w3w krb5kdc[12252](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.3.89.195@KRBTEST.COM for oryx/127.3.89.254@KRBTEST.COM, Server not found in Kerberos database
Aug 09 15:36:28 dist-test-slave-2w3w krb5kdc[12252](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.3.89.195@KRBTEST.COM for oryx/127.3.89.254@KRBTEST.COM, Server not found in Kerberos database
I20260809 15:36:28.343230 13367 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:36:28.337253 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.89.254:39259 (local address 127.3.89.195:59543)
0809 15:36:28.337534 (+ 281us) negotiation.cc:107] Waiting for socket to connect
0809 15:36:28.337547 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0809 15:36:28.337680 (+ 133us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 15:36:28.337953 (+ 273us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 15:36:28.337956 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 15:36:28.338170 (+ 214us) client_negotiation.cc:189] Negotiated authn=SASL
0809 15:36:28.338261 (+ 91us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:36:28.338267 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:36:28.339142 (+ 875us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:36:28.339147 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:36:28.339735 (+ 588us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:36:28.339780 (+ 45us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:36:28.340259 (+ 479us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:36:28.340262 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:36:28.340349 (+ 87us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:36:28.341063 (+ 714us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 15:36:28.341075 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0809 15:36:28.343098 (+ 2023us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.3.89.254:39259: Server oryx/127.3.89.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":216,"thread_start_us":92,"threads_started":1}
W20260809 15:36:28.343356 13365 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 15:36:28.337285 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.89.195:59543 (local address 127.3.89.254:39259)
0809 15:36:28.337433 (+ 148us) server_negotiation.cc:206] Beginning negotiation
0809 15:36:28.337437 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 15:36:28.337621 (+ 184us) server_negotiation.cc:407] Connection header received
0809 15:36:28.337785 (+ 164us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 15:36:28.337788 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 15:36:28.337843 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 15:36:28.337928 (+ 85us) server_negotiation.cc:226] Negotiated authn=SASL
0809 15:36:28.338378 (+ 450us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:36:28.339031 (+ 653us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:36:28.339900 (+ 869us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:36:28.340143 (+ 243us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:36:28.340256 (+ 113us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:36:28.343237 (+ 2981us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.3.89.195:59543: BlockingRecv error: recv got EOF from 127.3.89.195:59543 (error 108)
Metrics: {"server-negotiator.queue_time_us":50}
Aug 09 15:36:28 dist-test-slave-2w3w krb5kdc[12252](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.3.89.193@KRBTEST.COM for oryx/127.3.89.254@KRBTEST.COM, Server not found in Kerberos database
Aug 09 15:36:28 dist-test-slave-2w3w krb5kdc[12252](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.3.89.193@KRBTEST.COM for oryx/127.3.89.254@KRBTEST.COM, Server not found in Kerberos database
I20260809 15:36:28.985353 13368 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:36:28.979488 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.89.254:39259 (local address 127.3.89.193:33889)
0809 15:36:28.979760 (+ 272us) negotiation.cc:107] Waiting for socket to connect
0809 15:36:28.979774 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0809 15:36:28.979867 (+ 93us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 15:36:28.980108 (+ 241us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 15:36:28.980110 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 15:36:28.980315 (+ 205us) client_negotiation.cc:189] Negotiated authn=SASL
0809 15:36:28.980444 (+ 129us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:36:28.980453 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:36:28.981557 (+ 1104us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:36:28.981563 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:36:28.982149 (+ 586us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:36:28.982158 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:36:28.982571 (+ 413us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:36:28.982573 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:36:28.982636 (+ 63us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:36:28.983275 (+ 639us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 15:36:28.983300 (+ 25us) client_negotiation.cc:656] calling sasl_client_start()
0809 15:36:28.985230 (+ 1930us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.3.89.254:39259: Server oryx/127.3.89.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":175,"thread_start_us":74,"threads_started":1}
W20260809 15:36:28.985620 13369 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 15:36:28.979579 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.89.193:33889 (local address 127.3.89.254:39259)
0809 15:36:28.979812 (+ 233us) server_negotiation.cc:206] Beginning negotiation
0809 15:36:28.979815 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0809 15:36:28.979828 (+ 13us) server_negotiation.cc:407] Connection header received
0809 15:36:28.979936 (+ 108us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 15:36:28.979938 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 15:36:28.979990 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 15:36:28.980067 (+ 77us) server_negotiation.cc:226] Negotiated authn=SASL
0809 15:36:28.980577 (+ 510us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:36:28.981440 (+ 863us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:36:28.982254 (+ 814us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:36:28.982475 (+ 221us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:36:28.982554 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:36:28.985365 (+ 2811us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.3.89.193:33889: BlockingRecv error: recv got EOF from 127.3.89.193:33889 (error 108)
Metrics: {"server-negotiator.queue_time_us":162,"thread_start_us":84,"threads_started":1}
Aug 09 15:36:29 dist-test-slave-2w3w krb5kdc[12252](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.3.89.194@KRBTEST.COM for oryx/127.3.89.254@KRBTEST.COM, Server not found in Kerberos database
Aug 09 15:36:29 dist-test-slave-2w3w krb5kdc[12252](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.3.89.194@KRBTEST.COM for oryx/127.3.89.254@KRBTEST.COM, Server not found in Kerberos database
I20260809 15:36:29.165051 13370 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:36:29.159462 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.89.254:39259 (local address 127.3.89.194:48947)
0809 15:36:29.159737 (+ 275us) negotiation.cc:107] Waiting for socket to connect
0809 15:36:29.159758 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0809 15:36:29.159858 (+ 100us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 15:36:29.160089 (+ 231us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 15:36:29.160091 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 15:36:29.160279 (+ 188us) client_negotiation.cc:189] Negotiated authn=SASL
0809 15:36:29.160372 (+ 93us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:36:29.160378 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:36:29.161097 (+ 719us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:36:29.161099 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:36:29.161661 (+ 562us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:36:29.161668 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:36:29.162063 (+ 395us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:36:29.162065 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:36:29.162128 (+ 63us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:36:29.162663 (+ 535us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 15:36:29.162675 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0809 15:36:29.164932 (+ 2257us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.3.89.254:39259: Server oryx/127.3.89.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":182,"thread_start_us":92,"threads_started":1}
W20260809 15:36:29.165207 13369 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 15:36:29.159534 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.89.194:48947 (local address 127.3.89.254:39259)
0809 15:36:29.159690 (+ 156us) server_negotiation.cc:206] Beginning negotiation
0809 15:36:29.159693 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0809 15:36:29.159833 (+ 140us) server_negotiation.cc:407] Connection header received
0809 15:36:29.159936 (+ 103us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 15:36:29.159938 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 15:36:29.159983 (+ 45us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 15:36:29.160055 (+ 72us) server_negotiation.cc:226] Negotiated authn=SASL
0809 15:36:29.160477 (+ 422us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:36:29.160982 (+ 505us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:36:29.161762 (+ 780us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:36:29.161971 (+ 209us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:36:29.162044 (+ 73us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:36:29.165055 (+ 3011us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.3.89.194:48947: BlockingRecv error: recv got EOF from 127.3.89.194:48947 (error 108)
Metrics: {"server-negotiator.queue_time_us":64}
Aug 09 15:36:29 dist-test-slave-2w3w krb5kdc[12252](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.3.89.195@KRBTEST.COM for oryx/127.3.89.254@KRBTEST.COM, Server not found in Kerberos database
Aug 09 15:36:29 dist-test-slave-2w3w krb5kdc[12252](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.3.89.195@KRBTEST.COM for oryx/127.3.89.254@KRBTEST.COM, Server not found in Kerberos database
I20260809 15:36:29.349757 13371 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:36:29.344062 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.89.254:39259 (local address 127.3.89.195:41839)
0809 15:36:29.344386 (+ 324us) negotiation.cc:107] Waiting for socket to connect
0809 15:36:29.344399 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0809 15:36:29.344505 (+ 106us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 15:36:29.344751 (+ 246us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 15:36:29.344754 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 15:36:29.344960 (+ 206us) client_negotiation.cc:189] Negotiated authn=SASL
0809 15:36:29.345060 (+ 100us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:36:29.345065 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:36:29.345897 (+ 832us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:36:29.345902 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:36:29.346758 (+ 856us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:36:29.346768 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:36:29.347267 (+ 499us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:36:29.347270 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:36:29.347337 (+ 67us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:36:29.347900 (+ 563us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 15:36:29.347921 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0809 15:36:29.349642 (+ 1721us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.3.89.254:39259: Server oryx/127.3.89.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":223,"thread_start_us":112,"threads_started":1}
W20260809 15:36:29.349918 13369 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 15:36:29.344112 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.89.195:41839 (local address 127.3.89.254:39259)
0809 15:36:29.344252 (+ 140us) server_negotiation.cc:206] Beginning negotiation
0809 15:36:29.344255 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0809 15:36:29.344497 (+ 242us) server_negotiation.cc:407] Connection header received
0809 15:36:29.344589 (+ 92us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 15:36:29.344591 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 15:36:29.344638 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 15:36:29.344710 (+ 72us) server_negotiation.cc:226] Negotiated authn=SASL
0809 15:36:29.345155 (+ 445us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:36:29.345769 (+ 614us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:36:29.346879 (+ 1110us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:36:29.347163 (+ 284us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:36:29.347249 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:36:29.349774 (+ 2525us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.3.89.195:41839: BlockingRecv error: recv got EOF from 127.3.89.195:41839 (error 108)
Metrics: {"server-negotiator.queue_time_us":52}
Aug 09 15:36:29 dist-test-slave-2w3w krb5kdc[12252](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.3.89.193@KRBTEST.COM for oryx/127.3.89.254@KRBTEST.COM, Server not found in Kerberos database
Aug 09 15:36:29 dist-test-slave-2w3w krb5kdc[12252](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.3.89.193@KRBTEST.COM for oryx/127.3.89.254@KRBTEST.COM, Server not found in Kerberos database
I20260809 15:36:29.991984 13372 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:36:29.986304 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.89.254:39259 (local address 127.3.89.193:51745)
0809 15:36:29.986804 (+ 500us) negotiation.cc:107] Waiting for socket to connect
0809 15:36:29.986820 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0809 15:36:29.986952 (+ 132us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 15:36:29.987325 (+ 373us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 15:36:29.987331 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 15:36:29.987598 (+ 267us) client_negotiation.cc:189] Negotiated authn=SASL
0809 15:36:29.987743 (+ 145us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:36:29.987752 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:36:29.988537 (+ 785us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:36:29.988539 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:36:29.989105 (+ 566us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:36:29.989113 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:36:29.989528 (+ 415us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:36:29.989530 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:36:29.989593 (+ 63us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:36:29.990074 (+ 481us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 15:36:29.990085 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0809 15:36:29.991861 (+ 1776us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.3.89.254:39259: Server oryx/127.3.89.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":418,"thread_start_us":83,"threads_started":1}
W20260809 15:36:29.992133 13373 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 15:36:29.986369 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.89.193:51745 (local address 127.3.89.254:39259)
0809 15:36:29.986727 (+ 358us) server_negotiation.cc:206] Beginning negotiation
0809 15:36:29.986738 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0809 15:36:29.986901 (+ 163us) server_negotiation.cc:407] Connection header received
0809 15:36:29.987101 (+ 200us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 15:36:29.987105 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 15:36:29.987165 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 15:36:29.987258 (+ 93us) server_negotiation.cc:226] Negotiated authn=SASL
0809 15:36:29.987871 (+ 613us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:36:29.988429 (+ 558us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:36:29.989203 (+ 774us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:36:29.989429 (+ 226us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:36:29.989506 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:36:29.992002 (+ 2496us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.3.89.193:51745: BlockingRecv error: recv got EOF from 127.3.89.193:51745 (error 108)
Metrics: {"server-negotiator.queue_time_us":201,"thread_start_us":67,"threads_started":1}
Aug 09 15:36:30 dist-test-slave-2w3w krb5kdc[12252](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.3.89.194@KRBTEST.COM for oryx/127.3.89.254@KRBTEST.COM, Server not found in Kerberos database
Aug 09 15:36:30 dist-test-slave-2w3w krb5kdc[12252](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.3.89.194@KRBTEST.COM for oryx/127.3.89.254@KRBTEST.COM, Server not found in Kerberos database
I20260809 15:36:30.171267 13374 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:36:30.165890 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.89.254:39259 (local address 127.3.89.194:33645)
0809 15:36:30.166174 (+ 284us) negotiation.cc:107] Waiting for socket to connect
0809 15:36:30.166187 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0809 15:36:30.166285 (+ 98us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 15:36:30.166538 (+ 253us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 15:36:30.166541 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 15:36:30.166723 (+ 182us) client_negotiation.cc:189] Negotiated authn=SASL
0809 15:36:30.166822 (+ 99us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:36:30.166827 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:36:30.167564 (+ 737us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:36:30.167566 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:36:30.168151 (+ 585us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:36:30.168159 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:36:30.168569 (+ 410us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:36:30.168571 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:36:30.168631 (+ 60us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:36:30.169181 (+ 550us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 15:36:30.169193 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0809 15:36:30.171097 (+ 1904us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.3.89.254:39259: Server oryx/127.3.89.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":199,"thread_start_us":86,"threads_started":1}
W20260809 15:36:30.171442 13373 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 15:36:30.166047 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.89.194:33645 (local address 127.3.89.254:39259)
0809 15:36:30.166230 (+ 183us) server_negotiation.cc:206] Beginning negotiation
0809 15:36:30.166234 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 15:36:30.166247 (+ 13us) server_negotiation.cc:407] Connection header received
0809 15:36:30.166373 (+ 126us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 15:36:30.166375 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 15:36:30.166418 (+ 43us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 15:36:30.166504 (+ 86us) server_negotiation.cc:226] Negotiated authn=SASL
0809 15:36:30.166920 (+ 416us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:36:30.167472 (+ 552us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:36:30.168254 (+ 782us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:36:30.168476 (+ 222us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:36:30.168551 (+ 75us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:36:30.171324 (+ 2773us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.3.89.194:33645: BlockingRecv error: recv got EOF from 127.3.89.194:33645 (error 108)
Metrics: {"server-negotiator.queue_time_us":84}
Aug 09 15:36:30 dist-test-slave-2w3w krb5kdc[12252](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.3.89.195@KRBTEST.COM for oryx/127.3.89.254@KRBTEST.COM, Server not found in Kerberos database
Aug 09 15:36:30 dist-test-slave-2w3w krb5kdc[12252](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.3.89.195@KRBTEST.COM for oryx/127.3.89.254@KRBTEST.COM, Server not found in Kerberos database
I20260809 15:36:30.356215 13375 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:36:30.350607 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.89.254:39259 (local address 127.3.89.195:47115)
0809 15:36:30.350953 (+ 346us) negotiation.cc:107] Waiting for socket to connect
0809 15:36:30.350986 (+ 33us) client_negotiation.cc:174] Beginning negotiation
0809 15:36:30.351120 (+ 134us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 15:36:30.351394 (+ 274us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 15:36:30.351397 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 15:36:30.351606 (+ 209us) client_negotiation.cc:189] Negotiated authn=SASL
0809 15:36:30.351696 (+ 90us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:36:30.351701 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:36:30.352458 (+ 757us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:36:30.352463 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:36:30.353002 (+ 539us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:36:30.353009 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:36:30.353459 (+ 450us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:36:30.353462 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:36:30.353545 (+ 83us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:36:30.354087 (+ 542us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 15:36:30.354099 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0809 15:36:30.356031 (+ 1932us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.3.89.254:39259: Server oryx/127.3.89.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":264,"thread_start_us":82,"threads_started":1}
W20260809 15:36:30.356274 13373 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 15:36:30.350725 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.89.195:47115 (local address 127.3.89.254:39259)
0809 15:36:30.350913 (+ 188us) server_negotiation.cc:206] Beginning negotiation
0809 15:36:30.350918 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 15:36:30.351062 (+ 144us) server_negotiation.cc:407] Connection header received
0809 15:36:30.351219 (+ 157us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 15:36:30.351223 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 15:36:30.351278 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 15:36:30.351369 (+ 91us) server_negotiation.cc:226] Negotiated authn=SASL
0809 15:36:30.351833 (+ 464us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:36:30.352343 (+ 510us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:36:30.353119 (+ 776us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:36:30.353346 (+ 227us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:36:30.353453 (+ 107us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:36:30.356165 (+ 2712us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.3.89.195:47115: BlockingRecv error: recv got EOF from 127.3.89.195:47115 (error 108)
Metrics: {"server-negotiator.queue_time_us":91}
Aug 09 15:36:30 dist-test-slave-2w3w krb5kdc[12252](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.3.89.193@KRBTEST.COM for oryx/127.3.89.254@KRBTEST.COM, Server not found in Kerberos database
Aug 09 15:36:30 dist-test-slave-2w3w krb5kdc[12252](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.3.89.193@KRBTEST.COM for oryx/127.3.89.254@KRBTEST.COM, Server not found in Kerberos database
I20260809 15:36:31.002243 13377 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:36:30.992722 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.89.254:39259 (local address 127.3.89.193:34441)
0809 15:36:30.993031 (+ 309us) negotiation.cc:107] Waiting for socket to connect
0809 15:36:30.993045 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0809 15:36:30.993134 (+ 89us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 15:36:30.993374 (+ 240us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 15:36:30.993376 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 15:36:30.993565 (+ 189us) client_negotiation.cc:189] Negotiated authn=SASL
0809 15:36:30.993664 (+ 99us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:36:30.993669 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:36:30.994427 (+ 758us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:36:30.994429 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:36:30.995080 (+ 651us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:36:30.995089 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:36:30.995639 (+ 550us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:36:30.995642 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:36:30.995730 (+ 88us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:36:31.000202 (+ 4472us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 15:36:31.000215 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0809 15:36:31.002119 (+ 1904us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.3.89.254:39259: Server oryx/127.3.89.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":210,"thread_start_us":117,"threads_started":1}
W20260809 15:36:31.002562 13378 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 15:36:30.992840 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.89.193:34441 (local address 127.3.89.254:39259)
0809 15:36:30.993154 (+ 314us) server_negotiation.cc:206] Beginning negotiation
0809 15:36:30.993157 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0809 15:36:30.993170 (+ 13us) server_negotiation.cc:407] Connection header received
0809 15:36:30.993205 (+ 35us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 15:36:30.993207 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 15:36:30.993246 (+ 39us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 15:36:30.993333 (+ 87us) server_negotiation.cc:226] Negotiated authn=SASL
0809 15:36:30.993776 (+ 443us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:36:30.994329 (+ 553us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:36:30.995203 (+ 874us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:36:30.995509 (+ 306us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:36:30.995622 (+ 113us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:36:31.002261 (+ 6639us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.3.89.193:34441: BlockingRecv error: recv got EOF from 127.3.89.193:34441 (error 108)
Metrics: {"server-negotiator.queue_time_us":241,"thread_start_us":111,"threads_started":1}
Aug 09 15:36:31 dist-test-slave-2w3w krb5kdc[12252](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.3.89.194@KRBTEST.COM for oryx/127.3.89.254@KRBTEST.COM, Server not found in Kerberos database
Aug 09 15:36:31 dist-test-slave-2w3w krb5kdc[12252](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.3.89.194@KRBTEST.COM for oryx/127.3.89.254@KRBTEST.COM, Server not found in Kerberos database
I20260809 15:36:31.177469 13379 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:36:31.172068 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.89.254:39259 (local address 127.3.89.194:35115)
0809 15:36:31.172340 (+ 272us) negotiation.cc:107] Waiting for socket to connect
0809 15:36:31.172353 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0809 15:36:31.172451 (+ 98us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 15:36:31.172703 (+ 252us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 15:36:31.172705 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 15:36:31.172890 (+ 185us) client_negotiation.cc:189] Negotiated authn=SASL
0809 15:36:31.172983 (+ 93us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:36:31.172988 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:36:31.173702 (+ 714us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:36:31.173704 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:36:31.174266 (+ 562us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:36:31.174273 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:36:31.174676 (+ 403us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:36:31.174679 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:36:31.174749 (+ 70us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:36:31.175225 (+ 476us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 15:36:31.175236 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0809 15:36:31.177266 (+ 2030us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.3.89.254:39259: Server oryx/127.3.89.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":189,"thread_start_us":97,"threads_started":1}
W20260809 15:36:31.177501 13378 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 15:36:31.172169 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.89.194:35115 (local address 127.3.89.254:39259)
0809 15:36:31.172334 (+ 165us) server_negotiation.cc:206] Beginning negotiation
0809 15:36:31.172337 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0809 15:36:31.172423 (+ 86us) server_negotiation.cc:407] Connection header received
0809 15:36:31.172541 (+ 118us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 15:36:31.172544 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 15:36:31.172593 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 15:36:31.172665 (+ 72us) server_negotiation.cc:226] Negotiated authn=SASL
0809 15:36:31.173079 (+ 414us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:36:31.173610 (+ 531us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:36:31.174370 (+ 760us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:36:31.174581 (+ 211us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:36:31.174656 (+ 75us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:36:31.177389 (+ 2733us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.3.89.194:35115: BlockingRecv error: recv got EOF from 127.3.89.194:35115 (error 108)
Metrics: {"server-negotiator.queue_time_us":54}
Aug 09 15:36:31 dist-test-slave-2w3w krb5kdc[12252](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.3.89.195@KRBTEST.COM for oryx/127.3.89.254@KRBTEST.COM, Server not found in Kerberos database
Aug 09 15:36:31 dist-test-slave-2w3w krb5kdc[12252](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.3.89.195@KRBTEST.COM for oryx/127.3.89.254@KRBTEST.COM, Server not found in Kerberos database
I20260809 15:36:31.363279 13380 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:36:31.357052 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.89.254:39259 (local address 127.3.89.195:36095)
0809 15:36:31.357385 (+ 333us) negotiation.cc:107] Waiting for socket to connect
0809 15:36:31.357401 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0809 15:36:31.357567 (+ 166us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 15:36:31.357850 (+ 283us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 15:36:31.357852 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 15:36:31.358083 (+ 231us) client_negotiation.cc:189] Negotiated authn=SASL
0809 15:36:31.358177 (+ 94us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:36:31.358182 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:36:31.359125 (+ 943us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:36:31.359131 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:36:31.359917 (+ 786us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:36:31.359927 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:36:31.360379 (+ 452us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:36:31.360382 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:36:31.360453 (+ 71us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:36:31.360959 (+ 506us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 15:36:31.360972 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0809 15:36:31.363110 (+ 2138us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.3.89.254:39259: Server oryx/127.3.89.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":246,"thread_start_us":101,"threads_started":1}
W20260809 15:36:31.363386 13378 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 15:36:31.357237 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.89.195:36095 (local address 127.3.89.254:39259)
0809 15:36:31.357385 (+ 148us) server_negotiation.cc:206] Beginning negotiation
0809 15:36:31.357397 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0809 15:36:31.357504 (+ 107us) server_negotiation.cc:407] Connection header received
0809 15:36:31.357667 (+ 163us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 15:36:31.357671 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 15:36:31.357722 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 15:36:31.357822 (+ 100us) server_negotiation.cc:226] Negotiated authn=SASL
0809 15:36:31.358298 (+ 476us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:36:31.358961 (+ 663us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:36:31.360023 (+ 1062us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:36:31.360247 (+ 224us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:36:31.360331 (+ 84us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:36:31.363259 (+ 2928us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.3.89.195:36095: BlockingRecv error: recv got EOF from 127.3.89.195:36095 (error 108)
Metrics: {"server-negotiator.queue_time_us":60}
Aug 09 15:36:32 dist-test-slave-2w3w krb5kdc[12252](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.3.89.193@KRBTEST.COM for oryx/127.3.89.254@KRBTEST.COM, Server not found in Kerberos database
Aug 09 15:36:32 dist-test-slave-2w3w krb5kdc[12252](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.3.89.193@KRBTEST.COM for oryx/127.3.89.254@KRBTEST.COM, Server not found in Kerberos database
I20260809 15:36:32.009152 13381 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:36:32.003259 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.89.254:39259 (local address 127.3.89.193:56161)
0809 15:36:32.003603 (+ 344us) negotiation.cc:107] Waiting for socket to connect
0809 15:36:32.003616 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0809 15:36:32.003701 (+ 85us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 15:36:32.003996 (+ 295us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 15:36:32.003998 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 15:36:32.004186 (+ 188us) client_negotiation.cc:189] Negotiated authn=SASL
0809 15:36:32.004279 (+ 93us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:36:32.004284 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:36:32.005310 (+ 1026us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:36:32.005313 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:36:32.005984 (+ 671us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:36:32.005994 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:36:32.006492 (+ 498us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:36:32.006496 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:36:32.006581 (+ 85us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:36:32.007140 (+ 559us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 15:36:32.007152 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0809 15:36:32.008980 (+ 1828us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.3.89.254:39259: Server oryx/127.3.89.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":232,"thread_start_us":90,"threads_started":1}
W20260809 15:36:32.009249 13382 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 15:36:32.003346 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.89.193:56161 (local address 127.3.89.254:39259)
0809 15:36:32.003642 (+ 296us) server_negotiation.cc:206] Beginning negotiation
0809 15:36:32.003645 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0809 15:36:32.003661 (+ 16us) server_negotiation.cc:407] Connection header received
0809 15:36:32.003820 (+ 159us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 15:36:32.003823 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 15:36:32.003863 (+ 40us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 15:36:32.003964 (+ 101us) server_negotiation.cc:226] Negotiated authn=SASL
0809 15:36:32.004381 (+ 417us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:36:32.005200 (+ 819us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:36:32.006122 (+ 922us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:36:32.006374 (+ 252us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:36:32.006486 (+ 112us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:36:32.009111 (+ 2625us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.3.89.193:56161: BlockingRecv error: recv got EOF from 127.3.89.193:56161 (error 108)
Metrics: {"server-negotiator.queue_time_us":228,"thread_start_us":130,"threads_started":1}
Aug 09 15:36:32 dist-test-slave-2w3w krb5kdc[12252](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.3.89.194@KRBTEST.COM for oryx/127.3.89.254@KRBTEST.COM, Server not found in Kerberos database
Aug 09 15:36:32 dist-test-slave-2w3w krb5kdc[12252](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.3.89.194@KRBTEST.COM for oryx/127.3.89.254@KRBTEST.COM, Server not found in Kerberos database
I20260809 15:36:32.183744 13383 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:36:32.178338 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.89.254:39259 (local address 127.3.89.194:37417)
0809 15:36:32.178626 (+ 288us) negotiation.cc:107] Waiting for socket to connect
0809 15:36:32.178639 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0809 15:36:32.178732 (+ 93us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 15:36:32.178982 (+ 250us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 15:36:32.178985 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 15:36:32.179170 (+ 185us) client_negotiation.cc:189] Negotiated authn=SASL
0809 15:36:32.179265 (+ 95us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:36:32.179270 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:36:32.179979 (+ 709us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:36:32.179982 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:36:32.180540 (+ 558us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:36:32.180548 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:36:32.180979 (+ 431us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:36:32.180982 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:36:32.181045 (+ 63us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:36:32.181520 (+ 475us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 15:36:32.181530 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0809 15:36:32.183566 (+ 2036us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.3.89.254:39259: Server oryx/127.3.89.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":204,"thread_start_us":125,"threads_started":1}
W20260809 15:36:32.183848 13382 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 15:36:32.178449 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.89.194:37417 (local address 127.3.89.254:39259)
0809 15:36:32.178599 (+ 150us) server_negotiation.cc:206] Beginning negotiation
0809 15:36:32.178602 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0809 15:36:32.178707 (+ 105us) server_negotiation.cc:407] Connection header received
0809 15:36:32.178800 (+ 93us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 15:36:32.178804 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 15:36:32.178847 (+ 43us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 15:36:32.178918 (+ 71us) server_negotiation.cc:226] Negotiated authn=SASL
0809 15:36:32.179366 (+ 448us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:36:32.179880 (+ 514us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:36:32.180645 (+ 765us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:36:32.180871 (+ 226us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:36:32.180950 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:36:32.183729 (+ 2779us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.3.89.194:37417: BlockingRecv error: recv got EOF from 127.3.89.194:37417 (error 108)
Metrics: {"server-negotiator.queue_time_us":54}
I20260809 15:36:32.255491 3431 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskm314YR/build/debug/bin/kudu with pid 12877
I20260809 15:36:32.263334 3431 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskm314YR/build/debug/bin/kudu with pid 13042
I20260809 15:36:32.269215 3431 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskm314YR/build/debug/bin/kudu with pid 13178
I20260809 15:36:32.275147 3431 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskm314YR/build/debug/bin/kudu with pid 12803
2026-08-09T15:36:32Z chronyd exiting
[ OK ] SecurityITest.TestMismatchingPrincipals (18001 ms)
[ RUN ] SecurityITest.TestRequireAuthenticationInsecureCluster
2026-08-09T15:36:32Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T15:36:32Z Disabled control of system clock
I20260809 15:36:32.307542 3431 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskm314YR/build/debug/bin/kudu
/tmp/dist-test-taskm314YR/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786289732823074-3431-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786289732823074-3431-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786289732823074-3431-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786289732823074-3431-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.3.89.254:37857
--webserver_interface=127.3.89.254
--webserver_port=0
--builtin_ntp_servers=127.3.89.212:44681
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.3.89.254:37857
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {}
W20260809 15:36:32.416725 13387 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 15:36:32.416965 13387 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 15:36:32.416998 13387 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 15:36:32.421025 13387 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 15:36:32.421082 13387 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 15:36:32.421097 13387 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 15:36:32.421113 13387 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 15:36:32.421130 13387 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 15:36:32.425942 13387 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.3.89.212:44681
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786289732823074-3431-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786289732823074-3431-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.3.89.254:37857
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.3.89.254:37857
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786289732823074-3431-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.3.89.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.13387
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786289732823074-3431-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 09 Aug 2026 14:46:55 UTC on ccycloud.quasar-yswkmf.root.comops.site
build id 4576540
I20260809 15:36:32.427040 13387 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 15:36:32.428419 13387 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 15:36:32.436565 13395 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 15:36:32.436606 13392 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 15:36:32.436606 13393 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 15:36:32.436583 13387 server_base.cc:1034] running on GCE node
I20260809 15:36:32.437405 13387 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 15:36:32.438485 13387 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 15:36:32.439672 13387 hybrid_clock.cc:648] HybridClock initialized: now 1786289792439657 us; error 39 us; skew 500 ppm
I20260809 15:36:32.441491 13387 webserver.cc:460] Webserver started at http://127.3.89.254:45059/ using document root <none> and password file <none>
I20260809 15:36:32.442066 13387 fs_manager.cc:359] Metadata directory not provided
I20260809 15:36:32.442157 13387 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 15:36:32.442395 13387 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 15:36:32.444494 13387 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786289732823074-3431-0/minicluster-data/master-0/data/instance:
uuid: "b5e73b28066345e9b4c7da1d190154c2"
format_stamp: "Formatted at 2026-08-09 15:36:32 on dist-test-slave-2w3w"
server_key: "d6fa9b22dd7397dad8c498e2c32fdb44"
server_key_iv: "0448b067e653dcfec00437d198c175aa"
server_key_version: "encryptionkey@0"
I20260809 15:36:32.445058 13387 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786289732823074-3431-0/minicluster-data/master-0/wal/instance:
uuid: "b5e73b28066345e9b4c7da1d190154c2"
format_stamp: "Formatted at 2026-08-09 15:36:32 on dist-test-slave-2w3w"
server_key: "d6fa9b22dd7397dad8c498e2c32fdb44"
server_key_iv: "0448b067e653dcfec00437d198c175aa"
server_key_version: "encryptionkey@0"
I20260809 15:36:32.448661 13387 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.005s sys 0.000s
I20260809 15:36:32.450862 13401 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 15:36:32.452258 13387 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260809 15:36:32.452380 13387 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786289732823074-3431-0/minicluster-data/master-0/data,/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786289732823074-3431-0/minicluster-data/master-0/wal
uuid: "b5e73b28066345e9b4c7da1d190154c2"
format_stamp: "Formatted at 2026-08-09 15:36:32 on dist-test-slave-2w3w"
server_key: "d6fa9b22dd7397dad8c498e2c32fdb44"
server_key_iv: "0448b067e653dcfec00437d198c175aa"
server_key_version: "encryptionkey@0"
I20260809 15:36:32.452494 13387 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786289732823074-3431-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786289732823074-3431-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786289732823074-3431-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 15:36:32.464177 13387 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 15:36:32.464808 13387 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 15:36:32.465014 13387 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 15:36:32.471683 13387 rpc_server.cc:307] RPC server started. Bound to: 127.3.89.254:37857
I20260809 15:36:32.471692 13456 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.3.89.254:37857 every 8 connection(s)
I20260809 15:36:32.472590 13387 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786289732823074-3431-0/minicluster-data/master-0/data/info.pb
I20260809 15:36:32.473060 3431 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskm314YR/build/debug/bin/kudu as pid 13387
I20260809 15:36:32.473191 3431 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786289732823074-3431-0/minicluster-data/master-0/wal/instance
I20260809 15:36:32.473572 3431 external_mini_cluster.cc:1428] Setting key fcd0b108f759bdf0f2eeb2c8e905f16e
I20260809 15:36:32.475646 13457 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 15:36:32.481645 13457 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b5e73b28066345e9b4c7da1d190154c2: Bootstrap starting.
I20260809 15:36:32.482815 13460 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:36:32.474858 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60560 (local address 127.3.89.254:37857)
0809 15:36:32.475500 (+ 642us) server_negotiation.cc:206] Beginning negotiation
0809 15:36:32.475516 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0809 15:36:32.475552 (+ 36us) server_negotiation.cc:407] Connection header received
0809 15:36:32.476256 (+ 704us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 15:36:32.476272 (+ 16us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 15:36:32.476658 (+ 386us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 15:36:32.476924 (+ 266us) server_negotiation.cc:226] Negotiated authn=SASL
0809 15:36:32.477242 (+ 318us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:36:32.478415 (+ 1173us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:36:32.479659 (+ 1244us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:36:32.480108 (+ 449us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:36:32.480225 (+ 117us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:36:32.481091 (+ 866us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 15:36:32.481095 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 15:36:32.481107 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 15:36:32.481474 (+ 367us) server_negotiation.cc:922] calling sasl_server_start()
0809 15:36:32.481528 (+ 54us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 15:36:32.481547 (+ 19us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 15:36:32.481623 (+ 76us) server_negotiation.cc:1035] Waiting for connection context
0809 15:36:32.481826 (+ 203us) server_negotiation.cc:299] Negotiation successful
0809 15:36:32.482135 (+ 309us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":319,"thread_start_us":188,"threads_started":1}
I20260809 15:36:32.483919 13457 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P b5e73b28066345e9b4c7da1d190154c2: Neither blocks nor log segments found. Creating new log.
I20260809 15:36:32.484766 13457 log.cc:824] T 00000000000000000000000000000000 P b5e73b28066345e9b4c7da1d190154c2: Log is configured to *not* fsync() on all Append() calls
I20260809 15:36:32.486630 13457 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b5e73b28066345e9b4c7da1d190154c2: No bootstrap required, opened a new log
I20260809 15:36:32.490268 13457 raft_consensus.cc:348] T 00000000000000000000000000000000 P b5e73b28066345e9b4c7da1d190154c2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b5e73b28066345e9b4c7da1d190154c2" member_type: VOTER last_known_addr { host: "127.3.89.254" port: 37857 } }
I20260809 15:36:32.490463 13457 raft_consensus.cc:374] T 00000000000000000000000000000000 P b5e73b28066345e9b4c7da1d190154c2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 15:36:32.490522 13457 raft_consensus.cc:729] T 00000000000000000000000000000000 P b5e73b28066345e9b4c7da1d190154c2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b5e73b28066345e9b4c7da1d190154c2, State: Initialized, Role: FOLLOWER
I20260809 15:36:32.490914 13457 consensus_queue.cc:260] T 00000000000000000000000000000000 P b5e73b28066345e9b4c7da1d190154c2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b5e73b28066345e9b4c7da1d190154c2" member_type: VOTER last_known_addr { host: "127.3.89.254" port: 37857 } }
I20260809 15:36:32.491109 13457 raft_consensus.cc:388] T 00000000000000000000000000000000 P b5e73b28066345e9b4c7da1d190154c2 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 15:36:32.491174 13457 raft_consensus.cc:482] T 00000000000000000000000000000000 P b5e73b28066345e9b4c7da1d190154c2 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 15:36:32.491299 13457 raft_consensus.cc:3037] T 00000000000000000000000000000000 P b5e73b28066345e9b4c7da1d190154c2 [term 0 FOLLOWER]: Advancing to term 1
I20260809 15:36:32.492343 13457 raft_consensus.cc:504] T 00000000000000000000000000000000 P b5e73b28066345e9b4c7da1d190154c2 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b5e73b28066345e9b4c7da1d190154c2" member_type: VOTER last_known_addr { host: "127.3.89.254" port: 37857 } }
I20260809 15:36:32.492656 13457 leader_election.cc:302] T 00000000000000000000000000000000 P b5e73b28066345e9b4c7da1d190154c2 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: b5e73b28066345e9b4c7da1d190154c2; no voters:
I20260809 15:36:32.492939 13457 leader_election.cc:288] T 00000000000000000000000000000000 P b5e73b28066345e9b4c7da1d190154c2 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 15:36:32.493052 13462 raft_consensus.cc:2781] T 00000000000000000000000000000000 P b5e73b28066345e9b4c7da1d190154c2 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 15:36:32.493321 13462 raft_consensus.cc:686] T 00000000000000000000000000000000 P b5e73b28066345e9b4c7da1d190154c2 [term 1 LEADER]: Becoming Leader. State: Replica: b5e73b28066345e9b4c7da1d190154c2, State: Running, Role: LEADER
I20260809 15:36:32.493641 13462 consensus_queue.cc:237] T 00000000000000000000000000000000 P b5e73b28066345e9b4c7da1d190154c2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b5e73b28066345e9b4c7da1d190154c2" member_type: VOTER last_known_addr { host: "127.3.89.254" port: 37857 } }
I20260809 15:36:32.494220 13457 sys_catalog.cc:564] T 00000000000000000000000000000000 P b5e73b28066345e9b4c7da1d190154c2 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 15:36:32.495379 13464 sys_catalog.cc:455] T 00000000000000000000000000000000 P b5e73b28066345e9b4c7da1d190154c2 [sys.catalog]: SysCatalogTable state changed. Reason: New leader b5e73b28066345e9b4c7da1d190154c2. Latest consensus state: current_term: 1 leader_uuid: "b5e73b28066345e9b4c7da1d190154c2" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b5e73b28066345e9b4c7da1d190154c2" member_type: VOTER last_known_addr { host: "127.3.89.254" port: 37857 } } }
I20260809 15:36:32.495510 13464 sys_catalog.cc:458] T 00000000000000000000000000000000 P b5e73b28066345e9b4c7da1d190154c2 [sys.catalog]: This master's current role is: LEADER
I20260809 15:36:32.495782 13463 sys_catalog.cc:455] T 00000000000000000000000000000000 P b5e73b28066345e9b4c7da1d190154c2 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "b5e73b28066345e9b4c7da1d190154c2" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b5e73b28066345e9b4c7da1d190154c2" member_type: VOTER last_known_addr { host: "127.3.89.254" port: 37857 } } }
I20260809 15:36:32.495878 13463 sys_catalog.cc:458] T 00000000000000000000000000000000 P b5e73b28066345e9b4c7da1d190154c2 [sys.catalog]: This master's current role is: LEADER
I20260809 15:36:32.496003 13472 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 15:36:32.498803 13472 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 15:36:32.504063 13472 catalog_manager.cc:1347] Generated new cluster ID: ac06ba36d287450b80011a57161ac60d
I20260809 15:36:32.504130 13472 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 15:36:32.509765 13472 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 15:36:32.510876 13472 catalog_manager.cc:1504] Loading token signing keys...
I20260809 15:36:32.517390 13472 catalog_manager.cc:5868] T 00000000000000000000000000000000 P b5e73b28066345e9b4c7da1d190154c2: Generated new TSK 0
I20260809 15:36:32.518105 13472 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260809 15:36:32.523500 3431 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskm314YR/build/debug/bin/kudu
/tmp/dist-test-taskm314YR/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786289732823074-3431-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786289732823074-3431-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786289732823074-3431-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786289732823074-3431-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.3.89.193:0
--local_ip_for_outbound_sockets=127.3.89.193
--webserver_interface=127.3.89.193
--webserver_port=0
--tserver_master_addrs=127.3.89.254:37857
--builtin_ntp_servers=127.3.89.212:44681
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {}
W20260809 15:36:32.636477 13482 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 15:36:32.636708 13482 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 15:36:32.636744 13482 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 15:36:32.640867 13482 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 15:36:32.640931 13482 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 15:36:32.640985 13482 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.3.89.193
I20260809 15:36:32.647754 13482 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.3.89.212:44681
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786289732823074-3431-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786289732823074-3431-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.3.89.193:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786289732823074-3431-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.3.89.193
--webserver_port=0
--tserver_master_addrs=127.3.89.254:37857
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.13482
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.3.89.193
--log_dir=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786289732823074-3431-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 09 Aug 2026 14:46:55 UTC on ccycloud.quasar-yswkmf.root.comops.site
build id 4576540
I20260809 15:36:32.649255 13482 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 15:36:32.651228 13482 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 15:36:32.662563 13490 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 15:36:32.662578 13487 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 15:36:32.662720 13488 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 15:36:32.663256 13482 server_base.cc:1034] running on GCE node
I20260809 15:36:32.663745 13482 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 15:36:32.664562 13482 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 15:36:32.665745 13482 hybrid_clock.cc:648] HybridClock initialized: now 1786289792665727 us; error 57 us; skew 500 ppm
I20260809 15:36:32.668116 13482 webserver.cc:460] Webserver started at http://127.3.89.193:39993/ using document root <none> and password file <none>
I20260809 15:36:32.668927 13482 fs_manager.cc:359] Metadata directory not provided
I20260809 15:36:32.669015 13482 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 15:36:32.669209 13482 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 15:36:32.671729 13482 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786289732823074-3431-0/minicluster-data/ts-0/data/instance:
uuid: "5c7103fcac094394a3de10f469439700"
format_stamp: "Formatted at 2026-08-09 15:36:32 on dist-test-slave-2w3w"
server_key: "4da0cdce54209bf2ed5c0403569a4e60"
server_key_iv: "bfe47a855ad35a9fbc89069190eb9861"
server_key_version: "encryptionkey@0"
I20260809 15:36:32.672397 13482 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786289732823074-3431-0/minicluster-data/ts-0/wal/instance:
uuid: "5c7103fcac094394a3de10f469439700"
format_stamp: "Formatted at 2026-08-09 15:36:32 on dist-test-slave-2w3w"
server_key: "4da0cdce54209bf2ed5c0403569a4e60"
server_key_iv: "bfe47a855ad35a9fbc89069190eb9861"
server_key_version: "encryptionkey@0"
I20260809 15:36:32.677548 13482 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260809 15:36:32.681015 13497 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 15:36:32.682535 13482 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260809 15:36:32.682663 13482 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786289732823074-3431-0/minicluster-data/ts-0/data,/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786289732823074-3431-0/minicluster-data/ts-0/wal
uuid: "5c7103fcac094394a3de10f469439700"
format_stamp: "Formatted at 2026-08-09 15:36:32 on dist-test-slave-2w3w"
server_key: "4da0cdce54209bf2ed5c0403569a4e60"
server_key_iv: "bfe47a855ad35a9fbc89069190eb9861"
server_key_version: "encryptionkey@0"
I20260809 15:36:32.682809 13482 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786289732823074-3431-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786289732823074-3431-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786289732823074-3431-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 15:36:32.696552 13482 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 15:36:32.697252 13482 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 15:36:32.697461 13482 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 15:36:32.698071 13482 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 15:36:32.699100 13482 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 15:36:32.699169 13482 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 15:36:32.699235 13482 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 15:36:32.699285 13482 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 15:36:32.707928 13482 rpc_server.cc:307] RPC server started. Bound to: 127.3.89.193:38165
I20260809 15:36:32.707957 13619 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.3.89.193:38165 every 8 connection(s)
I20260809 15:36:32.708828 13482 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786289732823074-3431-0/minicluster-data/ts-0/data/info.pb
I20260809 15:36:32.711313 3431 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskm314YR/build/debug/bin/kudu as pid 13482
I20260809 15:36:32.711441 3431 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786289732823074-3431-0/minicluster-data/ts-0/wal/instance
I20260809 15:36:32.711746 3431 external_mini_cluster.cc:1428] Setting key 678ae7e47e0ab1d8c7762e297cb0644a
I20260809 15:36:32.713798 3431 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskm314YR/build/debug/bin/kudu
/tmp/dist-test-taskm314YR/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786289732823074-3431-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786289732823074-3431-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786289732823074-3431-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786289732823074-3431-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.3.89.194:0
--local_ip_for_outbound_sockets=127.3.89.194
--webserver_interface=127.3.89.194
--webserver_port=0
--tserver_master_addrs=127.3.89.254:37857
--builtin_ntp_servers=127.3.89.212:44681
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {}
I20260809 15:36:32.719710 13460 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:36:32.710629 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.89.193:48445 (local address 127.3.89.254:37857)
0809 15:36:32.710796 (+ 167us) server_negotiation.cc:206] Beginning negotiation
0809 15:36:32.710800 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 15:36:32.712018 (+ 1218us) server_negotiation.cc:407] Connection header received
0809 15:36:32.713565 (+ 1547us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 15:36:32.713571 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 15:36:32.713637 (+ 66us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 15:36:32.713740 (+ 103us) server_negotiation.cc:226] Negotiated authn=SASL
0809 15:36:32.714590 (+ 850us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:36:32.715494 (+ 904us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:36:32.717317 (+ 1823us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:36:32.717744 (+ 427us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:36:32.717874 (+ 130us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:36:32.719079 (+ 1205us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 15:36:32.719086 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 15:36:32.719091 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 15:36:32.719109 (+ 18us) server_negotiation.cc:922] calling sasl_server_start()
0809 15:36:32.719139 (+ 30us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 15:36:32.719153 (+ 14us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 15:36:32.719243 (+ 90us) server_negotiation.cc:1035] Waiting for connection context
0809 15:36:32.719388 (+ 145us) server_negotiation.cc:299] Negotiation successful
0809 15:36:32.719494 (+ 106us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":72}
I20260809 15:36:32.720484 13622 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:36:32.711135 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.89.254:37857 (local address 127.3.89.193:48445)
0809 15:36:32.711853 (+ 718us) negotiation.cc:107] Waiting for socket to connect
0809 15:36:32.711905 (+ 52us) client_negotiation.cc:174] Beginning negotiation
0809 15:36:32.713366 (+ 1461us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 15:36:32.713886 (+ 520us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 15:36:32.713892 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 15:36:32.713970 (+ 78us) client_negotiation.cc:189] Negotiated authn=SASL
0809 15:36:32.714446 (+ 476us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:36:32.714459 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:36:32.715678 (+ 1219us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:36:32.715683 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:36:32.717190 (+ 1507us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:36:32.717201 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:36:32.717898 (+ 697us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:36:32.717901 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:36:32.718037 (+ 136us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:36:32.718736 (+ 699us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0809 15:36:32.718739 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0809 15:36:32.718811 (+ 72us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0809 15:36:32.718890 (+ 79us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 15:36:32.719242 (+ 352us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 15:36:32.719250 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 15:36:32.719254 (+ 4us) client_negotiation.cc:769] Sending connection context
0809 15:36:32.719352 (+ 98us) client_negotiation.cc:240] Negotiation successful
0809 15:36:32.719568 (+ 216us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":282,"thread_start_us":147,"threads_started":1}
I20260809 15:36:32.721912 13620 heartbeater.cc:347] Connected to a master server at 127.3.89.254:37857
I20260809 15:36:32.722195 13620 heartbeater.cc:464] Registering TS with master...
I20260809 15:36:32.722985 13620 heartbeater.cc:511] Master 127.3.89.254:37857 requested a full tablet report, sending...
I20260809 15:36:32.724687 13419 ts_manager.cc:194] Registered new tserver with Master: 5c7103fcac094394a3de10f469439700 (127.3.89.193:38165)
I20260809 15:36:32.725873 13419 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.3.89.193:48445
W20260809 15:36:32.828373 13623 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 15:36:32.828611 13623 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 15:36:32.828646 13623 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 15:36:32.832721 13623 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 15:36:32.832770 13623 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 15:36:32.832844 13623 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.3.89.194
I20260809 15:36:32.837387 13623 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.3.89.212:44681
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786289732823074-3431-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786289732823074-3431-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.3.89.194:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786289732823074-3431-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.3.89.194
--webserver_port=0
--tserver_master_addrs=127.3.89.254:37857
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.13623
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.3.89.194
--log_dir=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786289732823074-3431-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 09 Aug 2026 14:46:55 UTC on ccycloud.quasar-yswkmf.root.comops.site
build id 4576540
I20260809 15:36:32.838358 13623 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 15:36:32.839746 13623 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 15:36:32.850201 13631 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 15:36:32.850100 13628 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 15:36:32.850348 13623 server_base.cc:1034] running on GCE node
W20260809 15:36:32.850104 13629 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 15:36:32.850759 13623 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 15:36:32.851451 13623 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 15:36:32.852643 13623 hybrid_clock.cc:648] HybridClock initialized: now 1786289792852615 us; error 47 us; skew 500 ppm
I20260809 15:36:32.854449 13623 webserver.cc:460] Webserver started at http://127.3.89.194:35483/ using document root <none> and password file <none>
I20260809 15:36:32.855087 13623 fs_manager.cc:359] Metadata directory not provided
I20260809 15:36:32.855183 13623 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 15:36:32.855405 13623 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 15:36:32.857491 13623 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786289732823074-3431-0/minicluster-data/ts-1/data/instance:
uuid: "4996d03ec6b949198475f0a056d48302"
format_stamp: "Formatted at 2026-08-09 15:36:32 on dist-test-slave-2w3w"
server_key: "a1f6d889cc343a6c6de763dcea7ea4ef"
server_key_iv: "025c12539e4b7b24f9087d1b9b976c0c"
server_key_version: "encryptionkey@0"
I20260809 15:36:32.858057 13623 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786289732823074-3431-0/minicluster-data/ts-1/wal/instance:
uuid: "4996d03ec6b949198475f0a056d48302"
format_stamp: "Formatted at 2026-08-09 15:36:32 on dist-test-slave-2w3w"
server_key: "a1f6d889cc343a6c6de763dcea7ea4ef"
server_key_iv: "025c12539e4b7b24f9087d1b9b976c0c"
server_key_version: "encryptionkey@0"
I20260809 15:36:32.861596 13623 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.006s sys 0.000s
I20260809 15:36:32.864022 13637 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 15:36:32.865167 13623 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260809 15:36:32.865301 13623 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786289732823074-3431-0/minicluster-data/ts-1/data,/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786289732823074-3431-0/minicluster-data/ts-1/wal
uuid: "4996d03ec6b949198475f0a056d48302"
format_stamp: "Formatted at 2026-08-09 15:36:32 on dist-test-slave-2w3w"
server_key: "a1f6d889cc343a6c6de763dcea7ea4ef"
server_key_iv: "025c12539e4b7b24f9087d1b9b976c0c"
server_key_version: "encryptionkey@0"
I20260809 15:36:32.865409 13623 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786289732823074-3431-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786289732823074-3431-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786289732823074-3431-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 15:36:32.881968 13623 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 15:36:32.882675 13623 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 15:36:32.882882 13623 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 15:36:32.883523 13623 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 15:36:32.884485 13623 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 15:36:32.884555 13623 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 15:36:32.884629 13623 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 15:36:32.884683 13623 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 15:36:32.893549 13623 rpc_server.cc:307] RPC server started. Bound to: 127.3.89.194:35025
I20260809 15:36:32.893584 13757 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.3.89.194:35025 every 8 connection(s)
I20260809 15:36:32.894510 13623 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786289732823074-3431-0/minicluster-data/ts-1/data/info.pb
I20260809 15:36:32.901863 3431 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskm314YR/build/debug/bin/kudu as pid 13623
I20260809 15:36:32.901988 3431 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786289732823074-3431-0/minicluster-data/ts-1/wal/instance
I20260809 15:36:32.902346 3431 external_mini_cluster.cc:1428] Setting key 8bdcf2a3e61e104647cd49f6c0548ec5
I20260809 15:36:32.903889 13460 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:36:32.896458 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.89.194:38867 (local address 127.3.89.254:37857)
0809 15:36:32.896618 (+ 160us) server_negotiation.cc:206] Beginning negotiation
0809 15:36:32.896623 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 15:36:32.897677 (+ 1054us) server_negotiation.cc:407] Connection header received
0809 15:36:32.899121 (+ 1444us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 15:36:32.899124 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 15:36:32.899170 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 15:36:32.899247 (+ 77us) server_negotiation.cc:226] Negotiated authn=SASL
0809 15:36:32.900102 (+ 855us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:36:32.900617 (+ 515us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:36:32.901761 (+ 1144us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:36:32.902016 (+ 255us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:36:32.902144 (+ 128us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:36:32.903405 (+ 1261us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 15:36:32.903408 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 15:36:32.903409 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 15:36:32.903420 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0809 15:36:32.903439 (+ 19us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 15:36:32.903447 (+ 8us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 15:36:32.903511 (+ 64us) server_negotiation.cc:1035] Waiting for connection context
0809 15:36:32.903687 (+ 176us) server_negotiation.cc:299] Negotiation successful
0809 15:36:32.903750 (+ 63us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
I20260809 15:36:32.904467 3431 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskm314YR/build/debug/bin/kudu
/tmp/dist-test-taskm314YR/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786289732823074-3431-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786289732823074-3431-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786289732823074-3431-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786289732823074-3431-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.3.89.195:0
--local_ip_for_outbound_sockets=127.3.89.195
--webserver_interface=127.3.89.195
--webserver_port=0
--tserver_master_addrs=127.3.89.254:37857
--builtin_ntp_servers=127.3.89.212:44681
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {}
I20260809 15:36:32.904795 13760 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:36:32.896870 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.89.254:37857 (local address 127.3.89.194:38867)
0809 15:36:32.897509 (+ 639us) negotiation.cc:107] Waiting for socket to connect
0809 15:36:32.897548 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0809 15:36:32.898885 (+ 1337us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 15:36:32.899406 (+ 521us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 15:36:32.899411 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 15:36:32.899486 (+ 75us) client_negotiation.cc:189] Negotiated authn=SASL
0809 15:36:32.899972 (+ 486us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:36:32.899984 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:36:32.900750 (+ 766us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:36:32.900752 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:36:32.901612 (+ 860us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:36:32.901621 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:36:32.902199 (+ 578us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:36:32.902204 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:36:32.902412 (+ 208us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:36:32.903114 (+ 702us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0809 15:36:32.903121 (+ 7us) client_negotiation.cc:656] calling sasl_client_start()
0809 15:36:32.903173 (+ 52us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0809 15:36:32.903249 (+ 76us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 15:36:32.903540 (+ 291us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 15:36:32.903549 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 15:36:32.903555 (+ 6us) client_negotiation.cc:769] Sending connection context
0809 15:36:32.903685 (+ 130us) client_negotiation.cc:240] Negotiation successful
0809 15:36:32.903961 (+ 276us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":264,"thread_start_us":112,"threads_started":1}
I20260809 15:36:32.905882 13758 heartbeater.cc:347] Connected to a master server at 127.3.89.254:37857
I20260809 15:36:32.906122 13758 heartbeater.cc:464] Registering TS with master...
I20260809 15:36:32.906658 13758 heartbeater.cc:511] Master 127.3.89.254:37857 requested a full tablet report, sending...
I20260809 15:36:32.907927 13419 ts_manager.cc:194] Registered new tserver with Master: 4996d03ec6b949198475f0a056d48302 (127.3.89.194:35025)
I20260809 15:36:32.908499 13419 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.3.89.194:38867
W20260809 15:36:33.016300 13761 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 15:36:33.016574 13761 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 15:36:33.016641 13761 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 15:36:33.020912 13761 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 15:36:33.020987 13761 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 15:36:33.021070 13761 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.3.89.195
I20260809 15:36:33.025653 13761 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.3.89.212:44681
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786289732823074-3431-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786289732823074-3431-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.3.89.195:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786289732823074-3431-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.3.89.195
--webserver_port=0
--tserver_master_addrs=127.3.89.254:37857
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.13761
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.3.89.195
--log_dir=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786289732823074-3431-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 09 Aug 2026 14:46:55 UTC on ccycloud.quasar-yswkmf.root.comops.site
build id 4576540
I20260809 15:36:33.026690 13761 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 15:36:33.028069 13761 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 15:36:33.037343 13766 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 15:36:33.037379 13769 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 15:36:33.037351 13767 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 15:36:33.037635 13761 server_base.cc:1034] running on GCE node
I20260809 15:36:33.038059 13761 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 15:36:33.038609 13761 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 15:36:33.039767 13761 hybrid_clock.cc:648] HybridClock initialized: now 1786289793039750 us; error 32 us; skew 500 ppm
I20260809 15:36:33.041510 13761 webserver.cc:460] Webserver started at http://127.3.89.195:43323/ using document root <none> and password file <none>
I20260809 15:36:33.042054 13761 fs_manager.cc:359] Metadata directory not provided
I20260809 15:36:33.042115 13761 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 15:36:33.042361 13761 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 15:36:33.044390 13761 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786289732823074-3431-0/minicluster-data/ts-2/data/instance:
uuid: "153cee9e70be42caaea0210d24bd121c"
format_stamp: "Formatted at 2026-08-09 15:36:33 on dist-test-slave-2w3w"
server_key: "970e69b975c74ed0a947c6b5f64465a2"
server_key_iv: "ee9006a276e7a765933850293c927469"
server_key_version: "encryptionkey@0"
I20260809 15:36:33.045126 13761 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786289732823074-3431-0/minicluster-data/ts-2/wal/instance:
uuid: "153cee9e70be42caaea0210d24bd121c"
format_stamp: "Formatted at 2026-08-09 15:36:33 on dist-test-slave-2w3w"
server_key: "970e69b975c74ed0a947c6b5f64465a2"
server_key_iv: "ee9006a276e7a765933850293c927469"
server_key_version: "encryptionkey@0"
I20260809 15:36:33.049953 13761 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.000s
I20260809 15:36:33.053242 13777 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 15:36:33.054693 13761 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.000s sys 0.002s
I20260809 15:36:33.054805 13761 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786289732823074-3431-0/minicluster-data/ts-2/data,/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786289732823074-3431-0/minicluster-data/ts-2/wal
uuid: "153cee9e70be42caaea0210d24bd121c"
format_stamp: "Formatted at 2026-08-09 15:36:33 on dist-test-slave-2w3w"
server_key: "970e69b975c74ed0a947c6b5f64465a2"
server_key_iv: "ee9006a276e7a765933850293c927469"
server_key_version: "encryptionkey@0"
I20260809 15:36:33.054888 13761 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786289732823074-3431-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786289732823074-3431-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786289732823074-3431-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 15:36:33.069653 13761 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 15:36:33.070190 13761 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 15:36:33.070325 13761 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 15:36:33.070793 13761 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 15:36:33.071763 13761 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 15:36:33.071815 13761 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 15:36:33.071851 13761 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 15:36:33.071867 13761 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 15:36:33.080638 13761 rpc_server.cc:307] RPC server started. Bound to: 127.3.89.195:42077
I20260809 15:36:33.080669 13895 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.3.89.195:42077 every 8 connection(s)
I20260809 15:36:33.081444 13761 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786289732823074-3431-0/minicluster-data/ts-2/data/info.pb
I20260809 15:36:33.081775 3431 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskm314YR/build/debug/bin/kudu as pid 13761
I20260809 15:36:33.081908 3431 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786289732823074-3431-0/minicluster-data/ts-2/wal/instance
I20260809 15:36:33.082278 3431 external_mini_cluster.cc:1428] Setting key bd2443935fed64fa836dec9fdc6e4f88
I20260809 15:36:33.090524 13460 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:36:33.083257 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.89.195:56913 (local address 127.3.89.254:37857)
0809 15:36:33.083414 (+ 157us) server_negotiation.cc:206] Beginning negotiation
0809 15:36:33.083419 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 15:36:33.084607 (+ 1188us) server_negotiation.cc:407] Connection header received
0809 15:36:33.086026 (+ 1419us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 15:36:33.086029 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 15:36:33.086074 (+ 45us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 15:36:33.086152 (+ 78us) server_negotiation.cc:226] Negotiated authn=SASL
0809 15:36:33.087159 (+ 1007us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:36:33.087760 (+ 601us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:36:33.088868 (+ 1108us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:36:33.089079 (+ 211us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:36:33.089159 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:36:33.090134 (+ 975us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 15:36:33.090137 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 15:36:33.090139 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 15:36:33.090150 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0809 15:36:33.090168 (+ 18us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 15:36:33.090175 (+ 7us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 15:36:33.090234 (+ 59us) server_negotiation.cc:1035] Waiting for connection context
0809 15:36:33.090368 (+ 134us) server_negotiation.cc:299] Negotiation successful
0809 15:36:33.090415 (+ 47us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":57}
I20260809 15:36:33.091364 13898 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:36:33.083749 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.89.254:37857 (local address 127.3.89.195:56913)
0809 15:36:33.084455 (+ 706us) negotiation.cc:107] Waiting for socket to connect
0809 15:36:33.084515 (+ 60us) client_negotiation.cc:174] Beginning negotiation
0809 15:36:33.085797 (+ 1282us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 15:36:33.086315 (+ 518us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 15:36:33.086324 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 15:36:33.086401 (+ 77us) client_negotiation.cc:189] Negotiated authn=SASL
0809 15:36:33.087012 (+ 611us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:36:33.087032 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:36:33.087890 (+ 858us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:36:33.087892 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:36:33.088753 (+ 861us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:36:33.088763 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:36:33.089207 (+ 444us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:36:33.089210 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:36:33.089342 (+ 132us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:36:33.089908 (+ 566us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0809 15:36:33.089909 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0809 15:36:33.089945 (+ 36us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0809 15:36:33.090001 (+ 56us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 15:36:33.090249 (+ 248us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 15:36:33.090255 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 15:36:33.090258 (+ 3us) client_negotiation.cc:769] Sending connection context
0809 15:36:33.090347 (+ 89us) client_negotiation.cc:240] Negotiation successful
0809 15:36:33.090553 (+ 206us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":279,"thread_start_us":145,"threads_started":1}
I20260809 15:36:33.092381 13896 heartbeater.cc:347] Connected to a master server at 127.3.89.254:37857
I20260809 15:36:33.092578 13896 heartbeater.cc:464] Registering TS with master...
I20260809 15:36:33.093101 13896 heartbeater.cc:511] Master 127.3.89.254:37857 requested a full tablet report, sending...
I20260809 15:36:33.094071 13419 ts_manager.cc:194] Registered new tserver with Master: 153cee9e70be42caaea0210d24bd121c (127.3.89.195:42077)
I20260809 15:36:33.094592 13419 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.3.89.195:56913
I20260809 15:36:33.096531 3431 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
W20260809 15:36:33.099483 13460 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 15:36:33.098443 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60572 (local address 127.3.89.254:37857)
0809 15:36:33.098577 (+ 134us) server_negotiation.cc:206] Beginning negotiation
0809 15:36:33.098581 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 15:36:33.098803 (+ 222us) server_negotiation.cc:407] Connection header received
0809 15:36:33.098888 (+ 85us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 15:36:33.098890 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 15:36:33.098926 (+ 36us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 15:36:33.099079 (+ 153us) server_negotiation.cc:226] Negotiated authn=SASL
0809 15:36:33.099336 (+ 257us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:60572: BlockingRecv error: recv got EOF from 127.0.0.1:60572 (error 108)
Metrics: {"server-negotiator.queue_time_us":56}
I20260809 15:36:33.099611 3431 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskm314YR/build/debug/bin/kudu with pid 13482
I20260809 15:36:33.110881 3431 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskm314YR/build/debug/bin/kudu with pid 13623
I20260809 15:36:33.116952 3431 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskm314YR/build/debug/bin/kudu with pid 13761
I20260809 15:36:33.123838 3431 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskm314YR/build/debug/bin/kudu with pid 13387
2026-08-09T15:36:33Z chronyd exiting
[ OK ] SecurityITest.TestRequireAuthenticationInsecureCluster (847 ms)
[ RUN ] SecurityITest.TestRequireEncryptionInsecureCluster
2026-08-09T15:36:33Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T15:36:33Z Disabled control of system clock
I20260809 15:36:33.154943 3431 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskm314YR/build/debug/bin/kudu
/tmp/dist-test-taskm314YR/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786289732823074-3431-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786289732823074-3431-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786289732823074-3431-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786289732823074-3431-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.3.89.254:34179
--webserver_interface=127.3.89.254
--webserver_port=0
--builtin_ntp_servers=127.3.89.212:43341
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.3.89.254:34179
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_encryption=disabled
--rpc_authentication=disabled with env {}
W20260809 15:36:33.262727 13907 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 15:36:33.262979 13907 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 15:36:33.263016 13907 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 15:36:33.266916 13907 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 15:36:33.266980 13907 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 15:36:33.266996 13907 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 15:36:33.267056 13907 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 15:36:33.267083 13907 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 15:36:33.271729 13907 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.3.89.212:43341
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786289732823074-3431-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786289732823074-3431-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.3.89.254:34179
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.3.89.254:34179
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786289732823074-3431-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.3.89.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.13907
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786289732823074-3431-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 09 Aug 2026 14:46:55 UTC on ccycloud.quasar-yswkmf.root.comops.site
build id 4576540
I20260809 15:36:33.272658 13907 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 15:36:33.273958 13907 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 15:36:33.282152 13916 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 15:36:33.282151 13913 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 15:36:33.282207 13907 server_base.cc:1034] running on GCE node
W20260809 15:36:33.282158 13914 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 15:36:33.283030 13907 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 15:36:33.284088 13907 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 15:36:33.285255 13907 hybrid_clock.cc:648] HybridClock initialized: now 1786289793285240 us; error 35 us; skew 500 ppm
I20260809 15:36:33.287184 13907 webserver.cc:460] Webserver started at http://127.3.89.254:42635/ using document root <none> and password file <none>
I20260809 15:36:33.287729 13907 fs_manager.cc:359] Metadata directory not provided
I20260809 15:36:33.287791 13907 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 15:36:33.288040 13907 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 15:36:33.290009 13907 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786289732823074-3431-0/minicluster-data/master-0/data/instance:
uuid: "109832c893d549bebf768890527dde87"
format_stamp: "Formatted at 2026-08-09 15:36:33 on dist-test-slave-2w3w"
server_key: "ef37ce7f9f3da048cb047d3b7f186a40"
server_key_iv: "dec09b4b5309511daebee51ca5bc8aea"
server_key_version: "encryptionkey@0"
I20260809 15:36:33.290592 13907 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786289732823074-3431-0/minicluster-data/master-0/wal/instance:
uuid: "109832c893d549bebf768890527dde87"
format_stamp: "Formatted at 2026-08-09 15:36:33 on dist-test-slave-2w3w"
server_key: "ef37ce7f9f3da048cb047d3b7f186a40"
server_key_iv: "dec09b4b5309511daebee51ca5bc8aea"
server_key_version: "encryptionkey@0"
I20260809 15:36:33.294340 13907 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.006s sys 0.000s
I20260809 15:36:33.296736 13922 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 15:36:33.297902 13907 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260809 15:36:33.298026 13907 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786289732823074-3431-0/minicluster-data/master-0/data,/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786289732823074-3431-0/minicluster-data/master-0/wal
uuid: "109832c893d549bebf768890527dde87"
format_stamp: "Formatted at 2026-08-09 15:36:33 on dist-test-slave-2w3w"
server_key: "ef37ce7f9f3da048cb047d3b7f186a40"
server_key_iv: "dec09b4b5309511daebee51ca5bc8aea"
server_key_version: "encryptionkey@0"
I20260809 15:36:33.298131 13907 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786289732823074-3431-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786289732823074-3431-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786289732823074-3431-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 15:36:33.302670 13907 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 15:36:33.303311 13907 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 15:36:33.303491 13907 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 15:36:33.310438 13907 rpc_server.cc:307] RPC server started. Bound to: 127.3.89.254:34179
I20260809 15:36:33.310452 13976 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.3.89.254:34179 every 8 connection(s)
I20260809 15:36:33.311372 13907 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786289732823074-3431-0/minicluster-data/master-0/data/info.pb
I20260809 15:36:33.314348 13977 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 15:36:33.319619 13977 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 109832c893d549bebf768890527dde87: Bootstrap starting.
I20260809 15:36:33.320379 3431 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskm314YR/build/debug/bin/kudu as pid 13907
I20260809 15:36:33.320504 3431 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786289732823074-3431-0/minicluster-data/master-0/wal/instance
I20260809 15:36:33.320827 3431 external_mini_cluster.cc:1428] Setting key c51de455b5178a62e12e57115532406a
I20260809 15:36:33.322510 13977 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 109832c893d549bebf768890527dde87: Neither blocks nor log segments found. Creating new log.
I20260809 15:36:33.323293 13977 log.cc:824] T 00000000000000000000000000000000 P 109832c893d549bebf768890527dde87: Log is configured to *not* fsync() on all Append() calls
I20260809 15:36:33.325193 13977 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 109832c893d549bebf768890527dde87: No bootstrap required, opened a new log
I20260809 15:36:33.326112 13980 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:36:33.321949 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39988 (local address 127.3.89.254:34179)
0809 15:36:33.322587 (+ 638us) server_negotiation.cc:206] Beginning negotiation
0809 15:36:33.322625 (+ 38us) server_negotiation.cc:399] Waiting for connection header
0809 15:36:33.322676 (+ 51us) server_negotiation.cc:407] Connection header received
0809 15:36:33.323341 (+ 665us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 15:36:33.323358 (+ 17us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 15:36:33.323776 (+ 418us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 15:36:33.324127 (+ 351us) server_negotiation.cc:226] Negotiated authn=SASL
0809 15:36:33.324927 (+ 800us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 15:36:33.324931 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 15:36:33.324942 (+ 11us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 15:36:33.324952 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0809 15:36:33.325006 (+ 54us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 15:36:33.325021 (+ 15us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 15:36:33.325137 (+ 116us) server_negotiation.cc:1035] Waiting for connection context
0809 15:36:33.325234 (+ 97us) server_negotiation.cc:299] Negotiation successful
0809 15:36:33.325491 (+ 257us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":297,"thread_start_us":140,"threads_started":1}
I20260809 15:36:33.328683 13977 raft_consensus.cc:348] T 00000000000000000000000000000000 P 109832c893d549bebf768890527dde87 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "109832c893d549bebf768890527dde87" member_type: VOTER last_known_addr { host: "127.3.89.254" port: 34179 } }
I20260809 15:36:33.328912 13977 raft_consensus.cc:374] T 00000000000000000000000000000000 P 109832c893d549bebf768890527dde87 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 15:36:33.328994 13977 raft_consensus.cc:729] T 00000000000000000000000000000000 P 109832c893d549bebf768890527dde87 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 109832c893d549bebf768890527dde87, State: Initialized, Role: FOLLOWER
I20260809 15:36:33.329384 13977 consensus_queue.cc:260] T 00000000000000000000000000000000 P 109832c893d549bebf768890527dde87 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "109832c893d549bebf768890527dde87" member_type: VOTER last_known_addr { host: "127.3.89.254" port: 34179 } }
I20260809 15:36:33.329542 13977 raft_consensus.cc:388] T 00000000000000000000000000000000 P 109832c893d549bebf768890527dde87 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 15:36:33.329639 13977 raft_consensus.cc:482] T 00000000000000000000000000000000 P 109832c893d549bebf768890527dde87 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 15:36:33.329756 13977 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 109832c893d549bebf768890527dde87 [term 0 FOLLOWER]: Advancing to term 1
I20260809 15:36:33.330597 13977 raft_consensus.cc:504] T 00000000000000000000000000000000 P 109832c893d549bebf768890527dde87 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "109832c893d549bebf768890527dde87" member_type: VOTER last_known_addr { host: "127.3.89.254" port: 34179 } }
I20260809 15:36:33.330922 13977 leader_election.cc:302] T 00000000000000000000000000000000 P 109832c893d549bebf768890527dde87 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 109832c893d549bebf768890527dde87; no voters:
I20260809 15:36:33.331274 13977 leader_election.cc:288] T 00000000000000000000000000000000 P 109832c893d549bebf768890527dde87 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 15:36:33.331408 13982 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 109832c893d549bebf768890527dde87 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 15:36:33.331673 13982 raft_consensus.cc:686] T 00000000000000000000000000000000 P 109832c893d549bebf768890527dde87 [term 1 LEADER]: Becoming Leader. State: Replica: 109832c893d549bebf768890527dde87, State: Running, Role: LEADER
I20260809 15:36:33.331980 13982 consensus_queue.cc:237] T 00000000000000000000000000000000 P 109832c893d549bebf768890527dde87 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "109832c893d549bebf768890527dde87" member_type: VOTER last_known_addr { host: "127.3.89.254" port: 34179 } }
I20260809 15:36:33.332511 13977 sys_catalog.cc:564] T 00000000000000000000000000000000 P 109832c893d549bebf768890527dde87 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 15:36:33.333918 13984 sys_catalog.cc:455] T 00000000000000000000000000000000 P 109832c893d549bebf768890527dde87 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 109832c893d549bebf768890527dde87. Latest consensus state: current_term: 1 leader_uuid: "109832c893d549bebf768890527dde87" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "109832c893d549bebf768890527dde87" member_type: VOTER last_known_addr { host: "127.3.89.254" port: 34179 } } }
I20260809 15:36:33.333948 13983 sys_catalog.cc:455] T 00000000000000000000000000000000 P 109832c893d549bebf768890527dde87 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "109832c893d549bebf768890527dde87" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "109832c893d549bebf768890527dde87" member_type: VOTER last_known_addr { host: "127.3.89.254" port: 34179 } } }
I20260809 15:36:33.334050 13984 sys_catalog.cc:458] T 00000000000000000000000000000000 P 109832c893d549bebf768890527dde87 [sys.catalog]: This master's current role is: LEADER
I20260809 15:36:33.334062 13983 sys_catalog.cc:458] T 00000000000000000000000000000000 P 109832c893d549bebf768890527dde87 [sys.catalog]: This master's current role is: LEADER
W20260809 15:36:33.337374 13997 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 109832c893d549bebf768890527dde87: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260809 15:36:33.337473 13997 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260809 15:36:33.337553 13993 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 15:36:33.338279 13993 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 15:36:33.343401 13993 catalog_manager.cc:1347] Generated new cluster ID: 87377fbb6d474f40855a7cc7c9bff33b
I20260809 15:36:33.343474 13993 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 15:36:33.369685 13993 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 15:36:33.370005 13993 catalog_manager.cc:1504] Loading token signing keys...
I20260809 15:36:33.377226 13993 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 109832c893d549bebf768890527dde87: Generated new TSK 0
I20260809 15:36:33.377822 13993 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260809 15:36:33.384449 3431 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskm314YR/build/debug/bin/kudu
/tmp/dist-test-taskm314YR/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786289732823074-3431-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786289732823074-3431-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786289732823074-3431-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786289732823074-3431-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.3.89.193:0
--local_ip_for_outbound_sockets=127.3.89.193
--webserver_interface=127.3.89.193
--webserver_port=0
--tserver_master_addrs=127.3.89.254:34179
--builtin_ntp_servers=127.3.89.212:43341
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_encryption=disabled
--rpc_authentication=disabled with env {}
W20260809 15:36:33.494567 14001 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 15:36:33.494864 14001 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 15:36:33.494946 14001 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 15:36:33.498857 14001 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 15:36:33.498929 14001 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 15:36:33.499035 14001 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.3.89.193
I20260809 15:36:33.504608 14001 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.3.89.212:43341
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786289732823074-3431-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786289732823074-3431-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.3.89.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786289732823074-3431-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.3.89.193
--webserver_port=0
--tserver_master_addrs=127.3.89.254:34179
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.14001
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.3.89.193
--log_dir=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786289732823074-3431-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 09 Aug 2026 14:46:55 UTC on ccycloud.quasar-yswkmf.root.comops.site
build id 4576540
I20260809 15:36:33.505761 14001 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 15:36:33.507217 14001 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 15:36:33.516826 14007 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 15:36:33.516878 14009 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 15:36:33.516825 14006 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 15:36:33.517102 14001 server_base.cc:1034] running on GCE node
I20260809 15:36:33.517524 14001 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 15:36:33.518195 14001 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 15:36:33.519366 14001 hybrid_clock.cc:648] HybridClock initialized: now 1786289793519345 us; error 31 us; skew 500 ppm
I20260809 15:36:33.521332 14001 webserver.cc:460] Webserver started at http://127.3.89.193:45557/ using document root <none> and password file <none>
I20260809 15:36:33.521924 14001 fs_manager.cc:359] Metadata directory not provided
I20260809 15:36:33.521992 14001 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 15:36:33.522261 14001 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 15:36:33.524413 14001 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786289732823074-3431-0/minicluster-data/ts-0/data/instance:
uuid: "11897f2a2a4748ef997fe1bb53819a57"
format_stamp: "Formatted at 2026-08-09 15:36:33 on dist-test-slave-2w3w"
server_key: "7733f738dea62eef25c9da60ea023b36"
server_key_iv: "a4bf88b904502ebdef6567574c540554"
server_key_version: "encryptionkey@0"
I20260809 15:36:33.525015 14001 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786289732823074-3431-0/minicluster-data/ts-0/wal/instance:
uuid: "11897f2a2a4748ef997fe1bb53819a57"
format_stamp: "Formatted at 2026-08-09 15:36:33 on dist-test-slave-2w3w"
server_key: "7733f738dea62eef25c9da60ea023b36"
server_key_iv: "a4bf88b904502ebdef6567574c540554"
server_key_version: "encryptionkey@0"
I20260809 15:36:33.528797 14001 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.005s sys 0.000s
I20260809 15:36:33.531373 14016 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 15:36:33.532615 14001 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260809 15:36:33.532769 14001 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786289732823074-3431-0/minicluster-data/ts-0/data,/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786289732823074-3431-0/minicluster-data/ts-0/wal
uuid: "11897f2a2a4748ef997fe1bb53819a57"
format_stamp: "Formatted at 2026-08-09 15:36:33 on dist-test-slave-2w3w"
server_key: "7733f738dea62eef25c9da60ea023b36"
server_key_iv: "a4bf88b904502ebdef6567574c540554"
server_key_version: "encryptionkey@0"
I20260809 15:36:33.532883 14001 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786289732823074-3431-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786289732823074-3431-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786289732823074-3431-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 15:36:33.537909 14001 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 15:36:33.538513 14001 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 15:36:33.538704 14001 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 15:36:33.539338 14001 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 15:36:33.540269 14001 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 15:36:33.540323 14001 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 15:36:33.540395 14001 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 15:36:33.540429 14001 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 15:36:33.549803 14001 rpc_server.cc:307] RPC server started. Bound to: 127.3.89.193:42417
I20260809 15:36:33.549823 14133 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.3.89.193:42417 every 8 connection(s)
I20260809 15:36:33.550690 14001 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786289732823074-3431-0/minicluster-data/ts-0/data/info.pb
I20260809 15:36:33.551229 3431 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskm314YR/build/debug/bin/kudu as pid 14001
I20260809 15:36:33.551344 3431 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786289732823074-3431-0/minicluster-data/ts-0/wal/instance
I20260809 15:36:33.551674 3431 external_mini_cluster.cc:1428] Setting key 5d19dd12f48c04c50fe3f04ac028111c
I20260809 15:36:33.553980 3431 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskm314YR/build/debug/bin/kudu
/tmp/dist-test-taskm314YR/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786289732823074-3431-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786289732823074-3431-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786289732823074-3431-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786289732823074-3431-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.3.89.194:0
--local_ip_for_outbound_sockets=127.3.89.194
--webserver_interface=127.3.89.194
--webserver_port=0
--tserver_master_addrs=127.3.89.254:34179
--builtin_ntp_servers=127.3.89.212:43341
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_encryption=disabled
--rpc_authentication=disabled with env {}
I20260809 15:36:33.558286 13980 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:36:33.552851 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.89.193:37425 (local address 127.3.89.254:34179)
0809 15:36:33.553003 (+ 152us) server_negotiation.cc:206] Beginning negotiation
0809 15:36:33.553009 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 15:36:33.554117 (+ 1108us) server_negotiation.cc:407] Connection header received
0809 15:36:33.555848 (+ 1731us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 15:36:33.555852 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 15:36:33.555892 (+ 40us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 15:36:33.555946 (+ 54us) server_negotiation.cc:226] Negotiated authn=SASL
0809 15:36:33.557697 (+ 1751us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 15:36:33.557703 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 15:36:33.557708 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 15:36:33.557720 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0809 15:36:33.557748 (+ 28us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 15:36:33.557761 (+ 13us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 15:36:33.557833 (+ 72us) server_negotiation.cc:1035] Waiting for connection context
0809 15:36:33.558094 (+ 261us) server_negotiation.cc:299] Negotiation successful
0809 15:36:33.558147 (+ 53us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":47}
I20260809 15:36:33.559895 14136 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:36:33.553254 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.89.254:34179 (local address 127.3.89.193:37425)
0809 15:36:33.553961 (+ 707us) negotiation.cc:107] Waiting for socket to connect
0809 15:36:33.554001 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0809 15:36:33.555588 (+ 1587us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 15:36:33.556335 (+ 747us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 15:36:33.556346 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 15:36:33.556496 (+ 150us) client_negotiation.cc:189] Negotiated authn=SASL
0809 15:36:33.557325 (+ 829us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0809 15:36:33.557328 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0809 15:36:33.557384 (+ 56us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0809 15:36:33.557496 (+ 112us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 15:36:33.557913 (+ 417us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 15:36:33.557924 (+ 11us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 15:36:33.557932 (+ 8us) client_negotiation.cc:769] Sending connection context
0809 15:36:33.558543 (+ 611us) client_negotiation.cc:240] Negotiation successful
0809 15:36:33.558824 (+ 281us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":305,"thread_start_us":142,"threads_started":1}
I20260809 15:36:33.563660 14134 heartbeater.cc:347] Connected to a master server at 127.3.89.254:34179
I20260809 15:36:33.563905 14134 heartbeater.cc:464] Registering TS with master...
I20260809 15:36:33.564467 14134 heartbeater.cc:511] Master 127.3.89.254:34179 requested a full tablet report, sending...
I20260809 15:36:33.566004 13940 ts_manager.cc:194] Registered new tserver with Master: 11897f2a2a4748ef997fe1bb53819a57 (127.3.89.193:42417)
W20260809 15:36:33.665192 14137 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 15:36:33.665462 14137 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 15:36:33.665540 14137 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 15:36:33.669476 14137 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 15:36:33.669555 14137 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 15:36:33.669646 14137 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.3.89.194
I20260809 15:36:33.674239 14137 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.3.89.212:43341
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786289732823074-3431-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786289732823074-3431-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.3.89.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786289732823074-3431-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.3.89.194
--webserver_port=0
--tserver_master_addrs=127.3.89.254:34179
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.14137
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.3.89.194
--log_dir=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786289732823074-3431-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 09 Aug 2026 14:46:55 UTC on ccycloud.quasar-yswkmf.root.comops.site
build id 4576540
I20260809 15:36:33.675276 14137 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 15:36:33.676651 14137 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 15:36:33.685839 14142 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 15:36:33.685818 14145 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 15:36:33.685808 14143 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 15:36:33.686246 14137 server_base.cc:1034] running on GCE node
I20260809 15:36:33.686700 14137 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 15:36:33.687368 14137 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 15:36:33.688553 14137 hybrid_clock.cc:648] HybridClock initialized: now 1786289793688524 us; error 42 us; skew 500 ppm
I20260809 15:36:33.690379 14137 webserver.cc:460] Webserver started at http://127.3.89.194:34765/ using document root <none> and password file <none>
I20260809 15:36:33.690951 14137 fs_manager.cc:359] Metadata directory not provided
I20260809 15:36:33.691103 14137 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 15:36:33.691337 14137 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 15:36:33.693329 14137 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786289732823074-3431-0/minicluster-data/ts-1/data/instance:
uuid: "d4d117bd237f440fb18b4fe62180c130"
format_stamp: "Formatted at 2026-08-09 15:36:33 on dist-test-slave-2w3w"
server_key: "bfdebeda9859271951d4b895bd0c1ee9"
server_key_iv: "0092a17d91851f6e5b9e3a59d84e4460"
server_key_version: "encryptionkey@0"
I20260809 15:36:33.693856 14137 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786289732823074-3431-0/minicluster-data/ts-1/wal/instance:
uuid: "d4d117bd237f440fb18b4fe62180c130"
format_stamp: "Formatted at 2026-08-09 15:36:33 on dist-test-slave-2w3w"
server_key: "bfdebeda9859271951d4b895bd0c1ee9"
server_key_iv: "0092a17d91851f6e5b9e3a59d84e4460"
server_key_version: "encryptionkey@0"
I20260809 15:36:33.697467 14137 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.002s sys 0.003s
I20260809 15:36:33.699849 14152 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 15:36:33.701005 14137 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260809 15:36:33.701097 14137 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786289732823074-3431-0/minicluster-data/ts-1/data,/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786289732823074-3431-0/minicluster-data/ts-1/wal
uuid: "d4d117bd237f440fb18b4fe62180c130"
format_stamp: "Formatted at 2026-08-09 15:36:33 on dist-test-slave-2w3w"
server_key: "bfdebeda9859271951d4b895bd0c1ee9"
server_key_iv: "0092a17d91851f6e5b9e3a59d84e4460"
server_key_version: "encryptionkey@0"
I20260809 15:36:33.701182 14137 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786289732823074-3431-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786289732823074-3431-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786289732823074-3431-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 15:36:33.705615 14137 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 15:36:33.706132 14137 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 15:36:33.706264 14137 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 15:36:33.706723 14137 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 15:36:33.707618 14137 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 15:36:33.707664 14137 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 15:36:33.707700 14137 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 15:36:33.707715 14137 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 15:36:33.716465 14137 rpc_server.cc:307] RPC server started. Bound to: 127.3.89.194:45791
I20260809 15:36:33.716480 14273 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.3.89.194:45791 every 8 connection(s)
I20260809 15:36:33.717304 14137 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786289732823074-3431-0/minicluster-data/ts-1/data/info.pb
I20260809 15:36:33.722057 3431 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskm314YR/build/debug/bin/kudu as pid 14137
I20260809 15:36:33.722208 3431 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786289732823074-3431-0/minicluster-data/ts-1/wal/instance
I20260809 15:36:33.722604 3431 external_mini_cluster.cc:1428] Setting key 95f494f0b2730d337bfe92bf972634c3
I20260809 15:36:33.723558 13980 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:36:33.719046 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.89.194:39373 (local address 127.3.89.254:34179)
0809 15:36:33.719172 (+ 126us) server_negotiation.cc:206] Beginning negotiation
0809 15:36:33.719176 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 15:36:33.720162 (+ 986us) server_negotiation.cc:407] Connection header received
0809 15:36:33.721475 (+ 1313us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 15:36:33.721479 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 15:36:33.721523 (+ 44us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 15:36:33.721611 (+ 88us) server_negotiation.cc:226] Negotiated authn=SASL
0809 15:36:33.723032 (+ 1421us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 15:36:33.723036 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 15:36:33.723039 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 15:36:33.723053 (+ 14us) server_negotiation.cc:922] calling sasl_server_start()
0809 15:36:33.723077 (+ 24us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 15:36:33.723090 (+ 13us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 15:36:33.723201 (+ 111us) server_negotiation.cc:1035] Waiting for connection context
0809 15:36:33.723388 (+ 187us) server_negotiation.cc:299] Negotiation successful
0809 15:36:33.723432 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":38}
I20260809 15:36:33.724581 14276 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:36:33.719416 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.89.254:34179 (local address 127.3.89.194:39373)
0809 15:36:33.720017 (+ 601us) negotiation.cc:107] Waiting for socket to connect
0809 15:36:33.720057 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0809 15:36:33.721259 (+ 1202us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 15:36:33.721819 (+ 560us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 15:36:33.721828 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 15:36:33.721939 (+ 111us) client_negotiation.cc:189] Negotiated authn=SASL
0809 15:36:33.722691 (+ 752us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0809 15:36:33.722693 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0809 15:36:33.722745 (+ 52us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0809 15:36:33.722828 (+ 83us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 15:36:33.723211 (+ 383us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 15:36:33.723221 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 15:36:33.723227 (+ 6us) client_negotiation.cc:769] Sending connection context
0809 15:36:33.723401 (+ 174us) client_negotiation.cc:240] Negotiation successful
0809 15:36:33.723603 (+ 202us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":238,"thread_start_us":101,"threads_started":1}
I20260809 15:36:33.724764 3431 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskm314YR/build/debug/bin/kudu
/tmp/dist-test-taskm314YR/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786289732823074-3431-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786289732823074-3431-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786289732823074-3431-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786289732823074-3431-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.3.89.195:0
--local_ip_for_outbound_sockets=127.3.89.195
--webserver_interface=127.3.89.195
--webserver_port=0
--tserver_master_addrs=127.3.89.254:34179
--builtin_ntp_servers=127.3.89.212:43341
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_encryption=disabled
--rpc_authentication=disabled with env {}
I20260809 15:36:33.725696 14274 heartbeater.cc:347] Connected to a master server at 127.3.89.254:34179
I20260809 15:36:33.725941 14274 heartbeater.cc:464] Registering TS with master...
I20260809 15:36:33.726339 14274 heartbeater.cc:511] Master 127.3.89.254:34179 requested a full tablet report, sending...
I20260809 15:36:33.727535 13940 ts_manager.cc:194] Registered new tserver with Master: d4d117bd237f440fb18b4fe62180c130 (127.3.89.194:45791)
W20260809 15:36:33.841148 14277 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 15:36:33.841370 14277 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 15:36:33.841408 14277 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 15:36:33.845161 14277 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 15:36:33.845203 14277 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 15:36:33.845256 14277 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.3.89.195
I20260809 15:36:33.850569 14277 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.3.89.212:43341
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786289732823074-3431-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786289732823074-3431-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.3.89.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786289732823074-3431-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.3.89.195
--webserver_port=0
--tserver_master_addrs=127.3.89.254:34179
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.14277
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.3.89.195
--log_dir=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786289732823074-3431-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 09 Aug 2026 14:46:55 UTC on ccycloud.quasar-yswkmf.root.comops.site
build id 4576540
I20260809 15:36:33.851590 14277 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 15:36:33.852849 14277 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 15:36:33.861994 14285 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 15:36:33.861985 14282 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 15:36:33.861982 14283 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 15:36:33.862088 14277 server_base.cc:1034] running on GCE node
I20260809 15:36:33.862607 14277 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 15:36:33.863272 14277 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 15:36:33.864450 14277 hybrid_clock.cc:648] HybridClock initialized: now 1786289793864447 us; error 35 us; skew 500 ppm
I20260809 15:36:33.866194 14277 webserver.cc:460] Webserver started at http://127.3.89.195:32897/ using document root <none> and password file <none>
I20260809 15:36:33.866739 14277 fs_manager.cc:359] Metadata directory not provided
I20260809 15:36:33.866834 14277 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 15:36:33.867113 14277 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 15:36:33.869107 14277 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786289732823074-3431-0/minicluster-data/ts-2/data/instance:
uuid: "a5ddfaa02b8c4b4197aa209882a62321"
format_stamp: "Formatted at 2026-08-09 15:36:33 on dist-test-slave-2w3w"
server_key: "f84a3f866555f35f2af2fad111636da5"
server_key_iv: "b9bc4f4fd586d362bbb787967bfa1170"
server_key_version: "encryptionkey@0"
I20260809 15:36:33.869695 14277 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786289732823074-3431-0/minicluster-data/ts-2/wal/instance:
uuid: "a5ddfaa02b8c4b4197aa209882a62321"
format_stamp: "Formatted at 2026-08-09 15:36:33 on dist-test-slave-2w3w"
server_key: "f84a3f866555f35f2af2fad111636da5"
server_key_iv: "b9bc4f4fd586d362bbb787967bfa1170"
server_key_version: "encryptionkey@0"
I20260809 15:36:33.873283 14277 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.005s sys 0.000s
I20260809 15:36:33.875684 14291 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 15:36:33.876952 14277 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260809 15:36:33.877084 14277 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786289732823074-3431-0/minicluster-data/ts-2/data,/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786289732823074-3431-0/minicluster-data/ts-2/wal
uuid: "a5ddfaa02b8c4b4197aa209882a62321"
format_stamp: "Formatted at 2026-08-09 15:36:33 on dist-test-slave-2w3w"
server_key: "f84a3f866555f35f2af2fad111636da5"
server_key_iv: "b9bc4f4fd586d362bbb787967bfa1170"
server_key_version: "encryptionkey@0"
I20260809 15:36:33.877188 14277 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786289732823074-3431-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786289732823074-3431-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786289732823074-3431-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 15:36:33.881594 14277 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 15:36:33.882210 14277 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 15:36:33.882414 14277 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 15:36:33.882951 14277 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 15:36:33.883859 14277 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 15:36:33.883905 14277 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 15:36:33.883978 14277 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 15:36:33.884016 14277 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 15:36:33.892338 14277 rpc_server.cc:307] RPC server started. Bound to: 127.3.89.195:34243
I20260809 15:36:33.892361 14410 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.3.89.195:34243 every 8 connection(s)
I20260809 15:36:33.893185 14277 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786289732823074-3431-0/minicluster-data/ts-2/data/info.pb
I20260809 15:36:33.899188 13980 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:36:33.895023 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.89.195:40479 (local address 127.3.89.254:34179)
0809 15:36:33.895148 (+ 125us) server_negotiation.cc:206] Beginning negotiation
0809 15:36:33.895151 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0809 15:36:33.896159 (+ 1008us) server_negotiation.cc:407] Connection header received
0809 15:36:33.897437 (+ 1278us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 15:36:33.897441 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 15:36:33.897485 (+ 44us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 15:36:33.897596 (+ 111us) server_negotiation.cc:226] Negotiated authn=SASL
0809 15:36:33.898654 (+ 1058us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 15:36:33.898658 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 15:36:33.898661 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 15:36:33.898672 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0809 15:36:33.898698 (+ 26us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 15:36:33.898712 (+ 14us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 15:36:33.898818 (+ 106us) server_negotiation.cc:1035] Waiting for connection context
0809 15:36:33.898995 (+ 177us) server_negotiation.cc:299] Negotiation successful
0809 15:36:33.899043 (+ 48us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":36}
I20260809 15:36:33.899967 14413 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:36:33.895385 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.89.254:34179 (local address 127.3.89.195:40479)
0809 15:36:33.896014 (+ 629us) negotiation.cc:107] Waiting for socket to connect
0809 15:36:33.896052 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0809 15:36:33.897227 (+ 1175us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 15:36:33.897751 (+ 524us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 15:36:33.897757 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 15:36:33.897836 (+ 79us) client_negotiation.cc:189] Negotiated authn=SASL
0809 15:36:33.898409 (+ 573us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0809 15:36:33.898411 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0809 15:36:33.898449 (+ 38us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0809 15:36:33.898510 (+ 61us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 15:36:33.898815 (+ 305us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 15:36:33.898823 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 15:36:33.898830 (+ 7us) client_negotiation.cc:769] Sending connection context
0809 15:36:33.899016 (+ 186us) client_negotiation.cc:240] Negotiation successful
0809 15:36:33.899259 (+ 243us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":256,"thread_start_us":112,"threads_started":1}
I20260809 15:36:33.901046 14411 heartbeater.cc:347] Connected to a master server at 127.3.89.254:34179
I20260809 15:36:33.901228 14411 heartbeater.cc:464] Registering TS with master...
I20260809 15:36:33.901582 14411 heartbeater.cc:511] Master 127.3.89.254:34179 requested a full tablet report, sending...
I20260809 15:36:33.902148 3431 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskm314YR/build/debug/bin/kudu as pid 14277
I20260809 15:36:33.902272 3431 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786289732823074-3431-0/minicluster-data/ts-2/wal/instance
I20260809 15:36:33.902596 3431 external_mini_cluster.cc:1428] Setting key d26015ac4f7fd97500d8d0fb3b49478f
I20260809 15:36:33.902590 13940 ts_manager.cc:194] Registered new tserver with Master: a5ddfaa02b8c4b4197aa209882a62321 (127.3.89.195:34243)
I20260809 15:36:33.904609 3431 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
W20260809 15:36:33.907197 13980 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 15:36:33.906441 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39996 (local address 127.3.89.254:34179)
0809 15:36:33.906563 (+ 122us) server_negotiation.cc:206] Beginning negotiation
0809 15:36:33.906569 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 15:36:33.906721 (+ 152us) server_negotiation.cc:407] Connection header received
0809 15:36:33.906794 (+ 73us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 15:36:33.906795 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 15:36:33.906821 (+ 26us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 15:36:33.906884 (+ 63us) server_negotiation.cc:226] Negotiated authn=SASL
0809 15:36:33.907116 (+ 232us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:39996: BlockingRecv error: recv got EOF from 127.0.0.1:39996 (error 108)
Metrics: {"server-negotiator.queue_time_us":54}
I20260809 15:36:33.907476 3431 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskm314YR/build/debug/bin/kudu with pid 14001
I20260809 15:36:33.919802 3431 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskm314YR/build/debug/bin/kudu with pid 14137
I20260809 15:36:33.928984 3431 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskm314YR/build/debug/bin/kudu with pid 14277
I20260809 15:36:33.935253 3431 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskm314YR/build/debug/bin/kudu with pid 13907
2026-08-09T15:36:33Z chronyd exiting
[ OK ] SecurityITest.TestRequireEncryptionInsecureCluster (811 ms)
[ RUN ] SecurityITest.TestRequireAuthenticationSecureCluster
Loading random data
Initializing database '/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786289732823074-3431-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 15:36:33 dist-test-slave-2w3w krb5kdc[14421](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 15:36:33 dist-test-slave-2w3w krb5kdc[14421](info): set up 2 sockets
Aug 09 15:36:33 dist-test-slave-2w3w krb5kdc[14421](info): commencing operation
krb5kdc: starting...
W20260809 15:36:35.010548 3431 mini_kdc.cc:121] Time spent starting KDC: real 1.053s user 0.000s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786289732823074-3431-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786289732823074-3431-0/krb5kdc/test-user.keytab.
Aug 09 15:36:35 dist-test-slave-2w3w krb5kdc[14421](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786289795, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T15:36:35Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T15:36:35Z Disabled control of system clock
WARNING: no policy specified for kudu/127.3.89.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.3.89.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.3.89.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786289732823074-3431-0/krb5kdc/kudu_127.3.89.254.keytab.
Entry for principal kudu/127.3.89.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786289732823074-3431-0/krb5kdc/kudu_127.3.89.254.keytab.
I20260809 15:36:35.135970 3431 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskm314YR/build/debug/bin/kudu
/tmp/dist-test-taskm314YR/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786289732823074-3431-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786289732823074-3431-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786289732823074-3431-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786289732823074-3431-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.3.89.254:38337
--webserver_interface=127.3.89.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786289732823074-3431-0/krb5kdc/kudu_127.3.89.254.keytab
--principal=kudu/127.3.89.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.3.89.212:34701
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.3.89.254:38337
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786289732823074-3431-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786289732823074-3431-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786289732823074-3431-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 15:36:35.246269 14434 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 15:36:35.246539 14434 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 15:36:35.246606 14434 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 15:36:35.250511 14434 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 15:36:35.250591 14434 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 15:36:35.250629 14434 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 15:36:35.250705 14434 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 15:36:35.250743 14434 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 15:36:35.255945 14434 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.3.89.212:34701
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786289732823074-3431-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786289732823074-3431-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.3.89.254:38337
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786289732823074-3431-0/krb5kdc/kudu_127.3.89.254.keytab
--principal=kudu/127.3.89.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.3.89.254:38337
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786289732823074-3431-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.3.89.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.14434
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786289732823074-3431-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 09 Aug 2026 14:46:55 UTC on ccycloud.quasar-yswkmf.root.comops.site
build id 4576540
I20260809 15:36:35.256937 14434 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 15:36:35.257897 14434 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 15:36:35.266191 14443 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 15:36:35.266265 14434 server_base.cc:1034] running on GCE node
W20260809 15:36:35.266215 14439 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 15:36:35.266181 14440 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 15:36:35.266942 14434 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 15:36:35.268023 14434 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 15:36:35.269209 14434 hybrid_clock.cc:648] HybridClock initialized: now 1786289795269191 us; error 30 us; skew 500 ppm
Aug 09 15:36:35 dist-test-slave-2w3w krb5kdc[14421](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786289795, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.89.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 15:36:35.271901 14434 init.cc:377] Logged in from keytab as kudu/127.3.89.254@KRBTEST.COM (short username kudu)
I20260809 15:36:35.272773 14434 webserver.cc:460] Webserver started at http://127.3.89.254:40757/ using document root <none> and password file <none>
I20260809 15:36:35.273301 14434 fs_manager.cc:359] Metadata directory not provided
I20260809 15:36:35.273360 14434 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 15:36:35.273578 14434 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 15:36:35.275571 14434 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786289732823074-3431-0/minicluster-data/master-0/data/instance:
uuid: "74b4eb3f44224d5582d2c1cfaaf0da55"
format_stamp: "Formatted at 2026-08-09 15:36:35 on dist-test-slave-2w3w"
server_key: "cba3efc4f753e1c422712d7c93aa8920"
server_key_iv: "f941227c69e6c7afc06b6b50b5eeb6a4"
server_key_version: "encryptionkey@0"
I20260809 15:36:35.276121 14434 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786289732823074-3431-0/minicluster-data/master-0/wal/instance:
uuid: "74b4eb3f44224d5582d2c1cfaaf0da55"
format_stamp: "Formatted at 2026-08-09 15:36:35 on dist-test-slave-2w3w"
server_key: "cba3efc4f753e1c422712d7c93aa8920"
server_key_iv: "f941227c69e6c7afc06b6b50b5eeb6a4"
server_key_version: "encryptionkey@0"
I20260809 15:36:35.279600 14434 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.003s sys 0.001s
I20260809 15:36:35.281869 14450 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 15:36:35.283087 14434 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.001s
I20260809 15:36:35.283215 14434 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786289732823074-3431-0/minicluster-data/master-0/data,/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786289732823074-3431-0/minicluster-data/master-0/wal
uuid: "74b4eb3f44224d5582d2c1cfaaf0da55"
format_stamp: "Formatted at 2026-08-09 15:36:35 on dist-test-slave-2w3w"
server_key: "cba3efc4f753e1c422712d7c93aa8920"
server_key_iv: "f941227c69e6c7afc06b6b50b5eeb6a4"
server_key_version: "encryptionkey@0"
I20260809 15:36:35.283329 14434 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786289732823074-3431-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786289732823074-3431-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786289732823074-3431-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 15:36:35.297328 14434 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 15:36:35.300027 14434 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 15:36:35.300223 14434 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 15:36:35.307207 14434 rpc_server.cc:307] RPC server started. Bound to: 127.3.89.254:38337
I20260809 15:36:35.307507 14506 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.3.89.254:38337 every 8 connection(s)
I20260809 15:36:35.308140 14434 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786289732823074-3431-0/minicluster-data/master-0/data/info.pb
I20260809 15:36:35.310648 14507 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 15:36:35.312415 3431 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskm314YR/build/debug/bin/kudu as pid 14434
I20260809 15:36:35.312548 3431 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786289732823074-3431-0/minicluster-data/master-0/wal/instance
I20260809 15:36:35.312834 3431 external_mini_cluster.cc:1428] Setting key e189c5eedd79cbee085b0756b980a30a
I20260809 15:36:35.316972 14507 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 74b4eb3f44224d5582d2c1cfaaf0da55: Bootstrap starting.
I20260809 15:36:35.319391 14507 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 74b4eb3f44224d5582d2c1cfaaf0da55: Neither blocks nor log segments found. Creating new log.
I20260809 15:36:35.320318 14507 log.cc:824] T 00000000000000000000000000000000 P 74b4eb3f44224d5582d2c1cfaaf0da55: Log is configured to *not* fsync() on all Append() calls
Aug 09 15:36:35 dist-test-slave-2w3w krb5kdc[14421](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786289795, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.3.89.254@KRBTEST.COM
I20260809 15:36:35.322257 14507 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 74b4eb3f44224d5582d2c1cfaaf0da55: No bootstrap required, opened a new log
I20260809 15:36:35.325894 14507 raft_consensus.cc:348] T 00000000000000000000000000000000 P 74b4eb3f44224d5582d2c1cfaaf0da55 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "74b4eb3f44224d5582d2c1cfaaf0da55" member_type: VOTER last_known_addr { host: "127.3.89.254" port: 38337 } }
I20260809 15:36:35.326156 14507 raft_consensus.cc:374] T 00000000000000000000000000000000 P 74b4eb3f44224d5582d2c1cfaaf0da55 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 15:36:35.326237 14507 raft_consensus.cc:729] T 00000000000000000000000000000000 P 74b4eb3f44224d5582d2c1cfaaf0da55 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 74b4eb3f44224d5582d2c1cfaaf0da55, State: Initialized, Role: FOLLOWER
I20260809 15:36:35.326669 14507 consensus_queue.cc:260] T 00000000000000000000000000000000 P 74b4eb3f44224d5582d2c1cfaaf0da55 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "74b4eb3f44224d5582d2c1cfaaf0da55" member_type: VOTER last_known_addr { host: "127.3.89.254" port: 38337 } }
I20260809 15:36:35.326828 14507 raft_consensus.cc:388] T 00000000000000000000000000000000 P 74b4eb3f44224d5582d2c1cfaaf0da55 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 15:36:35.326876 14510 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:36:35.314140 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36632 (local address 127.3.89.254:38337)
0809 15:36:35.314512 (+ 372us) server_negotiation.cc:206] Beginning negotiation
0809 15:36:35.314526 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0809 15:36:35.314560 (+ 34us) server_negotiation.cc:407] Connection header received
0809 15:36:35.315201 (+ 641us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 15:36:35.315217 (+ 16us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 15:36:35.315510 (+ 293us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 15:36:35.315761 (+ 251us) server_negotiation.cc:226] Negotiated authn=SASL
0809 15:36:35.316687 (+ 926us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:36:35.317681 (+ 994us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:36:35.318840 (+ 1159us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:36:35.319351 (+ 511us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:36:35.319465 (+ 114us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:36:35.322165 (+ 2700us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 15:36:35.322170 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 15:36:35.322181 (+ 11us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 15:36:35.322205 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0809 15:36:35.323925 (+ 1720us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 15:36:35.324367 (+ 442us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 15:36:35.324372 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 15:36:35.324376 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 15:36:35.324532 (+ 156us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 15:36:35.324835 (+ 303us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 15:36:35.324839 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 15:36:35.324840 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 15:36:35.325192 (+ 352us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 15:36:35.325415 (+ 223us) server_negotiation.cc:1035] Waiting for connection context
0809 15:36:35.325700 (+ 285us) server_negotiation.cc:299] Negotiation successful
0809 15:36:35.325967 (+ 267us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":238,"thread_start_us":117,"threads_started":1}
I20260809 15:36:35.326915 14507 raft_consensus.cc:482] T 00000000000000000000000000000000 P 74b4eb3f44224d5582d2c1cfaaf0da55 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 15:36:35.327055 14507 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 74b4eb3f44224d5582d2c1cfaaf0da55 [term 0 FOLLOWER]: Advancing to term 1
I20260809 15:36:35.327909 14507 raft_consensus.cc:504] T 00000000000000000000000000000000 P 74b4eb3f44224d5582d2c1cfaaf0da55 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "74b4eb3f44224d5582d2c1cfaaf0da55" member_type: VOTER last_known_addr { host: "127.3.89.254" port: 38337 } }
I20260809 15:36:35.328243 14507 leader_election.cc:302] T 00000000000000000000000000000000 P 74b4eb3f44224d5582d2c1cfaaf0da55 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 74b4eb3f44224d5582d2c1cfaaf0da55; no voters:
I20260809 15:36:35.328540 14507 leader_election.cc:288] T 00000000000000000000000000000000 P 74b4eb3f44224d5582d2c1cfaaf0da55 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 15:36:35.328671 14512 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 74b4eb3f44224d5582d2c1cfaaf0da55 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 15:36:35.328873 14512 raft_consensus.cc:686] T 00000000000000000000000000000000 P 74b4eb3f44224d5582d2c1cfaaf0da55 [term 1 LEADER]: Becoming Leader. State: Replica: 74b4eb3f44224d5582d2c1cfaaf0da55, State: Running, Role: LEADER
I20260809 15:36:35.329171 14512 consensus_queue.cc:237] T 00000000000000000000000000000000 P 74b4eb3f44224d5582d2c1cfaaf0da55 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "74b4eb3f44224d5582d2c1cfaaf0da55" member_type: VOTER last_known_addr { host: "127.3.89.254" port: 38337 } }
I20260809 15:36:35.329770 14507 sys_catalog.cc:564] T 00000000000000000000000000000000 P 74b4eb3f44224d5582d2c1cfaaf0da55 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 15:36:35.330235 14514 sys_catalog.cc:455] T 00000000000000000000000000000000 P 74b4eb3f44224d5582d2c1cfaaf0da55 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 74b4eb3f44224d5582d2c1cfaaf0da55. Latest consensus state: current_term: 1 leader_uuid: "74b4eb3f44224d5582d2c1cfaaf0da55" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "74b4eb3f44224d5582d2c1cfaaf0da55" member_type: VOTER last_known_addr { host: "127.3.89.254" port: 38337 } } }
I20260809 15:36:35.330413 14514 sys_catalog.cc:458] T 00000000000000000000000000000000 P 74b4eb3f44224d5582d2c1cfaaf0da55 [sys.catalog]: This master's current role is: LEADER
I20260809 15:36:35.330688 14513 sys_catalog.cc:455] T 00000000000000000000000000000000 P 74b4eb3f44224d5582d2c1cfaaf0da55 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "74b4eb3f44224d5582d2c1cfaaf0da55" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "74b4eb3f44224d5582d2c1cfaaf0da55" member_type: VOTER last_known_addr { host: "127.3.89.254" port: 38337 } } }
I20260809 15:36:35.330782 14513 sys_catalog.cc:458] T 00000000000000000000000000000000 P 74b4eb3f44224d5582d2c1cfaaf0da55 [sys.catalog]: This master's current role is: LEADER
I20260809 15:36:35.330853 14517 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 15:36:35.334185 14517 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 15:36:35.339834 14517 catalog_manager.cc:1347] Generated new cluster ID: 9a84dc92db6b4af0bd6192f1a84f1e1a
I20260809 15:36:35.339898 14517 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 15:36:35.346108 14517 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 15:36:35.347107 14517 catalog_manager.cc:1504] Loading token signing keys...
I20260809 15:36:35.354683 14517 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 74b4eb3f44224d5582d2c1cfaaf0da55: Generated new TSK 0
I20260809 15:36:35.355356 14517 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.3.89.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.3.89.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.3.89.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786289732823074-3431-0/krb5kdc/kudu_127.3.89.193.keytab.
Entry for principal kudu/127.3.89.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786289732823074-3431-0/krb5kdc/kudu_127.3.89.193.keytab.
I20260809 15:36:35.393397 3431 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskm314YR/build/debug/bin/kudu
/tmp/dist-test-taskm314YR/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786289732823074-3431-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786289732823074-3431-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786289732823074-3431-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786289732823074-3431-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.3.89.193:0
--local_ip_for_outbound_sockets=127.3.89.193
--webserver_interface=127.3.89.193
--webserver_port=0
--tserver_master_addrs=127.3.89.254:38337
--keytab_file=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786289732823074-3431-0/krb5kdc/kudu_127.3.89.193.keytab
--principal=kudu/127.3.89.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.3.89.212:34701
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786289732823074-3431-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786289732823074-3431-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786289732823074-3431-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 15:36:35.502256 14534 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 15:36:35.502530 14534 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 15:36:35.502607 14534 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 15:36:35.506531 14534 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 15:36:35.506616 14534 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 15:36:35.506708 14534 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.3.89.193
I20260809 15:36:35.511790 14534 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.3.89.212:34701
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786289732823074-3431-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786289732823074-3431-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786289732823074-3431-0/krb5kdc/kudu_127.3.89.193.keytab
--principal=kudu/127.3.89.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.3.89.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786289732823074-3431-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.3.89.193
--webserver_port=0
--tserver_master_addrs=127.3.89.254:38337
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.14534
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.3.89.193
--log_dir=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786289732823074-3431-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 09 Aug 2026 14:46:55 UTC on ccycloud.quasar-yswkmf.root.comops.site
build id 4576540
I20260809 15:36:35.512755 14534 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 15:36:35.513716 14534 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 15:36:35.523032 14540 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 15:36:35.523093 14539 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 15:36:35.523162 14534 server_base.cc:1034] running on GCE node
W20260809 15:36:35.523039 14542 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 15:36:35.523612 14534 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 15:36:35.524211 14534 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 15:36:35.525393 14534 hybrid_clock.cc:648] HybridClock initialized: now 1786289795525355 us; error 24 us; skew 500 ppm
Aug 09 15:36:35 dist-test-slave-2w3w krb5kdc[14421](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786289795, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.89.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 15:36:35.528095 14534 init.cc:377] Logged in from keytab as kudu/127.3.89.193@KRBTEST.COM (short username kudu)
I20260809 15:36:35.528892 14534 webserver.cc:460] Webserver started at http://127.3.89.193:37235/ using document root <none> and password file <none>
I20260809 15:36:35.529480 14534 fs_manager.cc:359] Metadata directory not provided
I20260809 15:36:35.529556 14534 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 15:36:35.529773 14534 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 15:36:35.531889 14534 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786289732823074-3431-0/minicluster-data/ts-0/data/instance:
uuid: "1690f2c02cb84e4caf44219967afd6ee"
format_stamp: "Formatted at 2026-08-09 15:36:35 on dist-test-slave-2w3w"
server_key: "9d359e0fe014e19aed264998ba423552"
server_key_iv: "34fbcd7dc6e1f5345028ade6d149844f"
server_key_version: "encryptionkey@0"
I20260809 15:36:35.532483 14534 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786289732823074-3431-0/minicluster-data/ts-0/wal/instance:
uuid: "1690f2c02cb84e4caf44219967afd6ee"
format_stamp: "Formatted at 2026-08-09 15:36:35 on dist-test-slave-2w3w"
server_key: "9d359e0fe014e19aed264998ba423552"
server_key_iv: "34fbcd7dc6e1f5345028ade6d149844f"
server_key_version: "encryptionkey@0"
I20260809 15:36:35.536011 14534 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.001s sys 0.004s
I20260809 15:36:35.538200 14549 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 15:36:35.539438 14534 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260809 15:36:35.539567 14534 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786289732823074-3431-0/minicluster-data/ts-0/data,/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786289732823074-3431-0/minicluster-data/ts-0/wal
uuid: "1690f2c02cb84e4caf44219967afd6ee"
format_stamp: "Formatted at 2026-08-09 15:36:35 on dist-test-slave-2w3w"
server_key: "9d359e0fe014e19aed264998ba423552"
server_key_iv: "34fbcd7dc6e1f5345028ade6d149844f"
server_key_version: "encryptionkey@0"
I20260809 15:36:35.539673 14534 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786289732823074-3431-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786289732823074-3431-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786289732823074-3431-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 15:36:35.551889 14534 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 15:36:35.554571 14534 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 15:36:35.554804 14534 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 15:36:35.555437 14534 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 15:36:35.556279 14534 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 15:36:35.556352 14534 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 15:36:35.556417 14534 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 15:36:35.556461 14534 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 15:36:35.565091 14534 rpc_server.cc:307] RPC server started. Bound to: 127.3.89.193:40909
I20260809 15:36:35.565135 14670 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.3.89.193:40909 every 8 connection(s)
I20260809 15:36:35.565943 14534 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786289732823074-3431-0/minicluster-data/ts-0/data/info.pb
I20260809 15:36:35.569723 3431 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskm314YR/build/debug/bin/kudu as pid 14534
I20260809 15:36:35.569815 3431 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786289732823074-3431-0/minicluster-data/ts-0/wal/instance
I20260809 15:36:35.570120 3431 external_mini_cluster.cc:1428] Setting key b71fb425ca3ecbb0c70c63b290681f78
Aug 09 15:36:35 dist-test-slave-2w3w krb5kdc[14421](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786289795, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.89.193@KRBTEST.COM for kudu/127.3.89.254@KRBTEST.COM
WARNING: no policy specified for kudu/127.3.89.194@KRBTEST.COM; defaulting to no policy
I20260809 15:36:35.580693 14510 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:36:35.567674 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.89.193:50637 (local address 127.3.89.254:38337)
0809 15:36:35.567865 (+ 191us) server_negotiation.cc:206] Beginning negotiation
0809 15:36:35.567869 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 15:36:35.568579 (+ 710us) server_negotiation.cc:407] Connection header received
0809 15:36:35.570055 (+ 1476us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 15:36:35.570059 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 15:36:35.570118 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 15:36:35.570217 (+ 99us) server_negotiation.cc:226] Negotiated authn=SASL
0809 15:36:35.571493 (+ 1276us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:36:35.572181 (+ 688us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:36:35.573670 (+ 1489us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:36:35.573896 (+ 226us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:36:35.573989 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:36:35.576974 (+ 2985us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 15:36:35.576977 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 15:36:35.576979 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 15:36:35.577002 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0809 15:36:35.578488 (+ 1486us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 15:36:35.579082 (+ 594us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 15:36:35.579086 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 15:36:35.579088 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 15:36:35.579171 (+ 83us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 15:36:35.579542 (+ 371us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 15:36:35.579545 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 15:36:35.579547 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 15:36:35.579727 (+ 180us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 15:36:35.579837 (+ 110us) server_negotiation.cc:1035] Waiting for connection context
0809 15:36:35.580350 (+ 513us) server_negotiation.cc:299] Negotiation successful
0809 15:36:35.580493 (+ 143us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":104}
I20260809 15:36:35.581465 14674 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:36:35.567980 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.89.254:38337 (local address 127.3.89.193:50637)
0809 15:36:35.568420 (+ 440us) negotiation.cc:107] Waiting for socket to connect
0809 15:36:35.568463 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0809 15:36:35.569806 (+ 1343us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 15:36:35.570387 (+ 581us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 15:36:35.570393 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 15:36:35.570798 (+ 405us) client_negotiation.cc:189] Negotiated authn=SASL
0809 15:36:35.571344 (+ 546us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:36:35.571359 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:36:35.572586 (+ 1227us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:36:35.572590 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:36:35.573541 (+ 951us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:36:35.573556 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:36:35.574053 (+ 497us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:36:35.574059 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:36:35.574289 (+ 230us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:36:35.574919 (+ 630us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 15:36:35.574955 (+ 36us) client_negotiation.cc:656] calling sasl_client_start()
0809 15:36:35.576794 (+ 1839us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 15:36:35.578622 (+ 1828us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 15:36:35.578628 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 15:36:35.578642 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0809 15:36:35.578908 (+ 266us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 15:36:35.579288 (+ 380us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 15:36:35.579291 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 15:36:35.579293 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 15:36:35.579434 (+ 141us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 15:36:35.579836 (+ 402us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 15:36:35.579842 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 15:36:35.580095 (+ 253us) client_negotiation.cc:769] Sending connection context
0809 15:36:35.580280 (+ 185us) client_negotiation.cc:240] Negotiation successful
0809 15:36:35.580640 (+ 360us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":257,"thread_start_us":104,"threads_started":1}
I20260809 15:36:35.582841 14671 heartbeater.cc:347] Connected to a master server at 127.3.89.254:38337
I20260809 15:36:35.583148 14671 heartbeater.cc:464] Registering TS with master...
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.3.89.194@KRBTEST.COM" created.
I20260809 15:36:35.583693 14671 heartbeater.cc:511] Master 127.3.89.254:38337 requested a full tablet report, sending...
I20260809 15:36:35.585434 14468 ts_manager.cc:194] Registered new tserver with Master: 1690f2c02cb84e4caf44219967afd6ee (127.3.89.193:40909)
I20260809 15:36:35.586723 14468 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.3.89.193@KRBTEST.COM'} at 127.3.89.193:50637
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.3.89.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786289732823074-3431-0/krb5kdc/kudu_127.3.89.194.keytab.
Entry for principal kudu/127.3.89.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786289732823074-3431-0/krb5kdc/kudu_127.3.89.194.keytab.
I20260809 15:36:35.600040 3431 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskm314YR/build/debug/bin/kudu
/tmp/dist-test-taskm314YR/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786289732823074-3431-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786289732823074-3431-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786289732823074-3431-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786289732823074-3431-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.3.89.194:0
--local_ip_for_outbound_sockets=127.3.89.194
--webserver_interface=127.3.89.194
--webserver_port=0
--tserver_master_addrs=127.3.89.254:38337
--keytab_file=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786289732823074-3431-0/krb5kdc/kudu_127.3.89.194.keytab
--principal=kudu/127.3.89.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.3.89.212:34701
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786289732823074-3431-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786289732823074-3431-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786289732823074-3431-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 15:36:35.707998 14677 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 15:36:35.708221 14677 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 15:36:35.708261 14677 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 15:36:35.712029 14677 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 15:36:35.712072 14677 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 15:36:35.712123 14677 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.3.89.194
I20260809 15:36:35.717060 14677 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.3.89.212:34701
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786289732823074-3431-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786289732823074-3431-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786289732823074-3431-0/krb5kdc/kudu_127.3.89.194.keytab
--principal=kudu/127.3.89.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.3.89.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786289732823074-3431-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.3.89.194
--webserver_port=0
--tserver_master_addrs=127.3.89.254:38337
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.14677
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.3.89.194
--log_dir=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786289732823074-3431-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 09 Aug 2026 14:46:55 UTC on ccycloud.quasar-yswkmf.root.comops.site
build id 4576540
I20260809 15:36:35.717900 14677 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 15:36:35.718776 14677 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 15:36:35.727958 14685 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 15:36:35.727988 14683 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 15:36:35.728137 14677 server_base.cc:1034] running on GCE node
W20260809 15:36:35.728313 14682 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 15:36:35.728703 14677 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 15:36:35.729308 14677 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 15:36:35.730466 14677 hybrid_clock.cc:648] HybridClock initialized: now 1786289795730447 us; error 23 us; skew 500 ppm
Aug 09 15:36:35 dist-test-slave-2w3w krb5kdc[14421](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786289795, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.89.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 15:36:35.733141 14677 init.cc:377] Logged in from keytab as kudu/127.3.89.194@KRBTEST.COM (short username kudu)
I20260809 15:36:35.733945 14677 webserver.cc:460] Webserver started at http://127.3.89.194:43113/ using document root <none> and password file <none>
I20260809 15:36:35.734423 14677 fs_manager.cc:359] Metadata directory not provided
I20260809 15:36:35.734488 14677 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 15:36:35.734642 14677 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 15:36:35.736621 14677 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786289732823074-3431-0/minicluster-data/ts-1/data/instance:
uuid: "f4f519de9f1c470993535452c3ba1f1b"
format_stamp: "Formatted at 2026-08-09 15:36:35 on dist-test-slave-2w3w"
server_key: "cdcdf6d19e6b9c5448a2135d0be91cc0"
server_key_iv: "110f9d9da776794afa46b95231675e69"
server_key_version: "encryptionkey@0"
I20260809 15:36:35.737124 14677 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786289732823074-3431-0/minicluster-data/ts-1/wal/instance:
uuid: "f4f519de9f1c470993535452c3ba1f1b"
format_stamp: "Formatted at 2026-08-09 15:36:35 on dist-test-slave-2w3w"
server_key: "cdcdf6d19e6b9c5448a2135d0be91cc0"
server_key_iv: "110f9d9da776794afa46b95231675e69"
server_key_version: "encryptionkey@0"
I20260809 15:36:35.740614 14677 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.005s sys 0.000s
I20260809 15:36:35.742796 14693 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 15:36:35.744081 14677 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260809 15:36:35.744202 14677 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786289732823074-3431-0/minicluster-data/ts-1/data,/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786289732823074-3431-0/minicluster-data/ts-1/wal
uuid: "f4f519de9f1c470993535452c3ba1f1b"
format_stamp: "Formatted at 2026-08-09 15:36:35 on dist-test-slave-2w3w"
server_key: "cdcdf6d19e6b9c5448a2135d0be91cc0"
server_key_iv: "110f9d9da776794afa46b95231675e69"
server_key_version: "encryptionkey@0"
I20260809 15:36:35.744303 14677 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786289732823074-3431-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786289732823074-3431-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786289732823074-3431-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 15:36:35.756285 14677 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 15:36:35.759385 14677 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 15:36:35.759578 14677 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 15:36:35.760162 14677 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 15:36:35.760954 14677 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 15:36:35.761000 14677 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 15:36:35.761066 14677 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 15:36:35.761101 14677 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 15:36:35.769459 14677 rpc_server.cc:307] RPC server started. Bound to: 127.3.89.194:44679
I20260809 15:36:35.769531 14811 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.3.89.194:44679 every 8 connection(s)
I20260809 15:36:35.770329 14677 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786289732823074-3431-0/minicluster-data/ts-1/data/info.pb
I20260809 15:36:35.776208 3431 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskm314YR/build/debug/bin/kudu as pid 14677
I20260809 15:36:35.776347 3431 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786289732823074-3431-0/minicluster-data/ts-1/wal/instance
I20260809 15:36:35.776639 3431 external_mini_cluster.cc:1428] Setting key e7e7dcfbb441b67e6288397721c336ea
Aug 09 15:36:35 dist-test-slave-2w3w krb5kdc[14421](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786289795, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.89.194@KRBTEST.COM for kudu/127.3.89.254@KRBTEST.COM
I20260809 15:36:35.784011 14510 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:36:35.772087 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.89.194:44065 (local address 127.3.89.254:38337)
0809 15:36:35.772232 (+ 145us) server_negotiation.cc:206] Beginning negotiation
0809 15:36:35.772235 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0809 15:36:35.773016 (+ 781us) server_negotiation.cc:407] Connection header received
0809 15:36:35.774226 (+ 1210us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 15:36:35.774228 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 15:36:35.774272 (+ 44us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 15:36:35.774348 (+ 76us) server_negotiation.cc:226] Negotiated authn=SASL
0809 15:36:35.775497 (+ 1149us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:36:35.776128 (+ 631us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:36:35.777431 (+ 1303us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:36:35.777687 (+ 256us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:36:35.777767 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:36:35.780422 (+ 2655us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 15:36:35.780433 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 15:36:35.780435 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 15:36:35.780465 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0809 15:36:35.781912 (+ 1447us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 15:36:35.782417 (+ 505us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 15:36:35.782421 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 15:36:35.782423 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 15:36:35.782507 (+ 84us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 15:36:35.782839 (+ 332us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 15:36:35.782844 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 15:36:35.782846 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 15:36:35.783112 (+ 266us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 15:36:35.783218 (+ 106us) server_negotiation.cc:1035] Waiting for connection context
0809 15:36:35.783693 (+ 475us) server_negotiation.cc:299] Negotiation successful
0809 15:36:35.783820 (+ 127us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":37,"server-negotiator.queue_time_us":62}
I20260809 15:36:35.784780 14814 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:36:35.772438 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.89.254:38337 (local address 127.3.89.194:44065)
0809 15:36:35.772866 (+ 428us) negotiation.cc:107] Waiting for socket to connect
0809 15:36:35.772905 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0809 15:36:35.774045 (+ 1140us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 15:36:35.774501 (+ 456us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 15:36:35.774506 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 15:36:35.774854 (+ 348us) client_negotiation.cc:189] Negotiated authn=SASL
0809 15:36:35.775369 (+ 515us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:36:35.775382 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:36:35.776245 (+ 863us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:36:35.776247 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:36:35.777315 (+ 1068us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:36:35.777328 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:36:35.777863 (+ 535us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:36:35.777867 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:36:35.778079 (+ 212us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:36:35.778678 (+ 599us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 15:36:35.778694 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0809 15:36:35.780260 (+ 1566us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 15:36:35.782036 (+ 1776us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 15:36:35.782043 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 15:36:35.782060 (+ 17us) client_negotiation.cc:760] calling sasl_client_step()
0809 15:36:35.782299 (+ 239us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 15:36:35.782599 (+ 300us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 15:36:35.782603 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 15:36:35.782605 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 15:36:35.782739 (+ 134us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 15:36:35.783217 (+ 478us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 15:36:35.783223 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 15:36:35.783450 (+ 227us) client_negotiation.cc:769] Sending connection context
0809 15:36:35.783647 (+ 197us) client_negotiation.cc:240] Negotiation successful
0809 15:36:35.783956 (+ 309us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":265,"thread_start_us":122,"threads_started":1}
I20260809 15:36:35.785928 14812 heartbeater.cc:347] Connected to a master server at 127.3.89.254:38337
I20260809 15:36:35.786183 14812 heartbeater.cc:464] Registering TS with master...
I20260809 15:36:35.786774 14812 heartbeater.cc:511] Master 127.3.89.254:38337 requested a full tablet report, sending...
WARNING: no policy specified for kudu/127.3.89.195@KRBTEST.COM; defaulting to no policy
I20260809 15:36:35.787963 14468 ts_manager.cc:194] Registered new tserver with Master: f4f519de9f1c470993535452c3ba1f1b (127.3.89.194:44679)
I20260809 15:36:35.788522 14468 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.3.89.194@KRBTEST.COM'} at 127.3.89.194:44065
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.3.89.195@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.3.89.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786289732823074-3431-0/krb5kdc/kudu_127.3.89.195.keytab.
Entry for principal kudu/127.3.89.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786289732823074-3431-0/krb5kdc/kudu_127.3.89.195.keytab.
I20260809 15:36:35.805972 3431 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskm314YR/build/debug/bin/kudu
/tmp/dist-test-taskm314YR/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786289732823074-3431-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786289732823074-3431-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786289732823074-3431-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786289732823074-3431-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.3.89.195:0
--local_ip_for_outbound_sockets=127.3.89.195
--webserver_interface=127.3.89.195
--webserver_port=0
--tserver_master_addrs=127.3.89.254:38337
--keytab_file=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786289732823074-3431-0/krb5kdc/kudu_127.3.89.195.keytab
--principal=kudu/127.3.89.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.3.89.212:34701
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786289732823074-3431-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786289732823074-3431-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786289732823074-3431-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 15:36:35.925942 14817 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 15:36:35.926168 14817 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 15:36:35.926247 14817 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 15:36:35.930518 14817 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 15:36:35.930565 14817 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 15:36:35.930660 14817 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.3.89.195
I20260809 15:36:35.935758 14817 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.3.89.212:34701
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786289732823074-3431-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786289732823074-3431-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786289732823074-3431-0/krb5kdc/kudu_127.3.89.195.keytab
--principal=kudu/127.3.89.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.3.89.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786289732823074-3431-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.3.89.195
--webserver_port=0
--tserver_master_addrs=127.3.89.254:38337
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.14817
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.3.89.195
--log_dir=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786289732823074-3431-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 09 Aug 2026 14:46:55 UTC on ccycloud.quasar-yswkmf.root.comops.site
build id 4576540
I20260809 15:36:35.936648 14817 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 15:36:35.937572 14817 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 15:36:35.946738 14822 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 15:36:35.946753 14823 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 15:36:35.946748 14825 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 15:36:35.946789 14817 server_base.cc:1034] running on GCE node
I20260809 15:36:35.947388 14817 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 15:36:35.948055 14817 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 15:36:35.949235 14817 hybrid_clock.cc:648] HybridClock initialized: now 1786289795949217 us; error 34 us; skew 500 ppm
Aug 09 15:36:35 dist-test-slave-2w3w krb5kdc[14421](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786289795, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.89.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 15:36:35.951990 14817 init.cc:377] Logged in from keytab as kudu/127.3.89.195@KRBTEST.COM (short username kudu)
I20260809 15:36:35.952814 14817 webserver.cc:460] Webserver started at http://127.3.89.195:42389/ using document root <none> and password file <none>
I20260809 15:36:35.953347 14817 fs_manager.cc:359] Metadata directory not provided
I20260809 15:36:35.953409 14817 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 15:36:35.953658 14817 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 15:36:35.955682 14817 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786289732823074-3431-0/minicluster-data/ts-2/data/instance:
uuid: "074f582a381c46bb9cdaab6945c79e4e"
format_stamp: "Formatted at 2026-08-09 15:36:35 on dist-test-slave-2w3w"
server_key: "65d6c75a26ccb999109f31cbbfc3111a"
server_key_iv: "2006f91a5956b02c42c086015b65c604"
server_key_version: "encryptionkey@0"
I20260809 15:36:35.956236 14817 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786289732823074-3431-0/minicluster-data/ts-2/wal/instance:
uuid: "074f582a381c46bb9cdaab6945c79e4e"
format_stamp: "Formatted at 2026-08-09 15:36:35 on dist-test-slave-2w3w"
server_key: "65d6c75a26ccb999109f31cbbfc3111a"
server_key_iv: "2006f91a5956b02c42c086015b65c604"
server_key_version: "encryptionkey@0"
I20260809 15:36:35.959765 14817 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.004s sys 0.000s
I20260809 15:36:35.962167 14833 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 15:36:35.963514 14817 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260809 15:36:35.963639 14817 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786289732823074-3431-0/minicluster-data/ts-2/data,/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786289732823074-3431-0/minicluster-data/ts-2/wal
uuid: "074f582a381c46bb9cdaab6945c79e4e"
format_stamp: "Formatted at 2026-08-09 15:36:35 on dist-test-slave-2w3w"
server_key: "65d6c75a26ccb999109f31cbbfc3111a"
server_key_iv: "2006f91a5956b02c42c086015b65c604"
server_key_version: "encryptionkey@0"
I20260809 15:36:35.963775 14817 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786289732823074-3431-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786289732823074-3431-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786289732823074-3431-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 15:36:35.973186 14817 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 15:36:35.975926 14817 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 15:36:35.976130 14817 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 15:36:35.976750 14817 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 15:36:35.977524 14817 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 15:36:35.977571 14817 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 15:36:35.977638 14817 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 15:36:35.977687 14817 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 15:36:35.986505 14817 rpc_server.cc:307] RPC server started. Bound to: 127.3.89.195:38177
I20260809 15:36:35.986543 14954 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.3.89.195:38177 every 8 connection(s)
I20260809 15:36:35.987457 14817 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786289732823074-3431-0/minicluster-data/ts-2/data/info.pb
I20260809 15:36:35.992651 3431 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskm314YR/build/debug/bin/kudu as pid 14817
I20260809 15:36:35.992761 3431 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786289732823074-3431-0/minicluster-data/ts-2/wal/instance
I20260809 15:36:35.993028 3431 external_mini_cluster.cc:1428] Setting key 4ffced700ce693b33ab51be195e93b30
Aug 09 15:36:35 dist-test-slave-2w3w krb5kdc[14421](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786289795, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.89.195@KRBTEST.COM for kudu/127.3.89.254@KRBTEST.COM
I20260809 15:36:36.000797 14510 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:36:35.989308 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.89.195:37889 (local address 127.3.89.254:38337)
0809 15:36:35.989448 (+ 140us) server_negotiation.cc:206] Beginning negotiation
0809 15:36:35.989451 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0809 15:36:35.990206 (+ 755us) server_negotiation.cc:407] Connection header received
0809 15:36:35.991554 (+ 1348us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 15:36:35.991557 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 15:36:35.991620 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 15:36:35.991724 (+ 104us) server_negotiation.cc:226] Negotiated authn=SASL
0809 15:36:35.992902 (+ 1178us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:36:35.993504 (+ 602us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:36:35.994824 (+ 1320us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:36:35.995157 (+ 333us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:36:35.995256 (+ 99us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:36:35.997681 (+ 2425us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 15:36:35.997683 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 15:36:35.997686 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 15:36:35.997709 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0809 15:36:35.998934 (+ 1225us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 15:36:35.999488 (+ 554us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 15:36:35.999491 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 15:36:35.999492 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 15:36:35.999574 (+ 82us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 15:36:35.999884 (+ 310us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 15:36:35.999886 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 15:36:35.999887 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 15:36:36.000083 (+ 196us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 15:36:36.000161 (+ 78us) server_negotiation.cc:1035] Waiting for connection context
0809 15:36:36.000579 (+ 418us) server_negotiation.cc:299] Negotiation successful
0809 15:36:36.000678 (+ 99us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":55}
I20260809 15:36:36.001536 14957 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:36:35.989608 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.89.254:38337 (local address 127.3.89.195:37889)
0809 15:36:35.990053 (+ 445us) negotiation.cc:107] Waiting for socket to connect
0809 15:36:35.990094 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0809 15:36:35.991339 (+ 1245us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 15:36:35.991891 (+ 552us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 15:36:35.991897 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 15:36:35.992244 (+ 347us) client_negotiation.cc:189] Negotiated authn=SASL
0809 15:36:35.992699 (+ 455us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:36:35.992759 (+ 60us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:36:35.993638 (+ 879us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:36:35.993641 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:36:35.994687 (+ 1046us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:36:35.994698 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:36:35.995318 (+ 620us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:36:35.995323 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:36:35.995483 (+ 160us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:36:35.996006 (+ 523us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 15:36:35.996021 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0809 15:36:35.997532 (+ 1511us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 15:36:35.999105 (+ 1573us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 15:36:35.999111 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 15:36:35.999124 (+ 13us) client_negotiation.cc:760] calling sasl_client_step()
0809 15:36:35.999379 (+ 255us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 15:36:35.999657 (+ 278us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 15:36:35.999659 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 15:36:35.999660 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0809 15:36:35.999791 (+ 131us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 15:36:36.000173 (+ 382us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 15:36:36.000178 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 15:36:36.000379 (+ 201us) client_negotiation.cc:769] Sending connection context
0809 15:36:36.000529 (+ 150us) client_negotiation.cc:240] Negotiation successful
0809 15:36:36.000759 (+ 230us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":269,"thread_start_us":126,"threads_started":1}
I20260809 15:36:36.002606 14955 heartbeater.cc:347] Connected to a master server at 127.3.89.254:38337
I20260809 15:36:36.002830 14955 heartbeater.cc:464] Registering TS with master...
I20260809 15:36:36.003333 14955 heartbeater.cc:511] Master 127.3.89.254:38337 requested a full tablet report, sending...
I20260809 15:36:36.004308 14468 ts_manager.cc:194] Registered new tserver with Master: 074f582a381c46bb9cdaab6945c79e4e (127.3.89.195:38177)
I20260809 15:36:36.004858 14468 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.3.89.195@KRBTEST.COM'} at 127.3.89.195:37889
I20260809 15:36:36.006923 3431 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260809 15:36:36.019639 14510 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:36:36.008772 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36634 (local address 127.3.89.254:38337)
0809 15:36:36.008912 (+ 140us) server_negotiation.cc:206] Beginning negotiation
0809 15:36:36.008916 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 15:36:36.009102 (+ 186us) server_negotiation.cc:407] Connection header received
0809 15:36:36.009188 (+ 86us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 15:36:36.009190 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 15:36:36.009222 (+ 32us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 15:36:36.009294 (+ 72us) server_negotiation.cc:226] Negotiated authn=SASL
0809 15:36:36.009903 (+ 609us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:36:36.010360 (+ 457us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:36:36.011158 (+ 798us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:36:36.011358 (+ 200us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:36:36.011435 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:36:36.016941 (+ 5506us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 15:36:36.016944 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 15:36:36.016945 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 15:36:36.016966 (+ 21us) server_negotiation.cc:922] calling sasl_server_start()
0809 15:36:36.018195 (+ 1229us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 15:36:36.018581 (+ 386us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 15:36:36.018584 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 15:36:36.018586 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 15:36:36.018655 (+ 69us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 15:36:36.018887 (+ 232us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 15:36:36.018889 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 15:36:36.018890 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 15:36:36.019135 (+ 245us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 15:36:36.019240 (+ 105us) server_negotiation.cc:1035] Waiting for connection context
0809 15:36:36.019447 (+ 207us) server_negotiation.cc:299] Negotiation successful
0809 15:36:36.019532 (+ 85us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":72}
I20260809 15:36:36.022195 14468 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:36634:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260809 15:36:36.025228 14468 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260809 15:36:36.040082 14967 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:36:36.033658 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52952 (local address 127.3.89.194:44679)
0809 15:36:36.034519 (+ 861us) server_negotiation.cc:206] Beginning negotiation
0809 15:36:36.034533 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0809 15:36:36.034559 (+ 26us) server_negotiation.cc:407] Connection header received
0809 15:36:36.034633 (+ 74us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 15:36:36.034638 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 15:36:36.034860 (+ 222us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 15:36:36.035592 (+ 732us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 15:36:36.035763 (+ 171us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:36:36.036899 (+ 1136us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:36:36.038507 (+ 1608us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:36:36.039355 (+ 848us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:36:36.039477 (+ 122us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:36:36.039631 (+ 154us) server_negotiation.cc:1035] Waiting for connection context
0809 15:36:36.039823 (+ 192us) server_negotiation.cc:299] Negotiation successful
0809 15:36:36.039931 (+ 108us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":754,"thread_start_us":133,"threads_started":1}
I20260809 15:36:36.040095 14966 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:36:36.033512 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.89.194:44679 (local address 127.0.0.1:52952)
0809 15:36:36.034053 (+ 541us) negotiation.cc:107] Waiting for socket to connect
0809 15:36:36.034098 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0809 15:36:36.034232 (+ 134us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 15:36:36.035336 (+ 1104us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 15:36:36.035342 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 15:36:36.035366 (+ 24us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 15:36:36.035670 (+ 304us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:36:36.035679 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:36:36.037077 (+ 1398us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:36:36.037083 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:36:36.038370 (+ 1287us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:36:36.038382 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:36:36.039488 (+ 1106us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:36:36.039492 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:36:36.039616 (+ 124us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:36:36.039658 (+ 42us) client_negotiation.cc:769] Sending connection context
0809 15:36:36.039780 (+ 122us) client_negotiation.cc:240] Negotiation successful
0809 15:36:36.039949 (+ 169us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":381,"mutex_wait_us":224,"spinlock_wait_cycles":768,"thread_start_us":81,"threads_started":1}
I20260809 15:36:36.040393 14970 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:36:36.034401 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59728 (local address 127.3.89.193:40909)
0809 15:36:36.034831 (+ 430us) server_negotiation.cc:206] Beginning negotiation
0809 15:36:36.034842 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0809 15:36:36.034864 (+ 22us) server_negotiation.cc:407] Connection header received
0809 15:36:36.034939 (+ 75us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 15:36:36.034944 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 15:36:36.035165 (+ 221us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 15:36:36.035464 (+ 299us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 15:36:36.035711 (+ 247us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:36:36.036908 (+ 1197us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:36:36.038701 (+ 1793us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:36:36.039511 (+ 810us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:36:36.039953 (+ 442us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:36:36.040090 (+ 137us) server_negotiation.cc:1035] Waiting for connection context
0809 15:36:36.040144 (+ 54us) server_negotiation.cc:299] Negotiation successful
0809 15:36:36.040255 (+ 111us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":303,"thread_start_us":124,"threads_started":1}
I20260809 15:36:36.040519 14969 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:36:36.034194 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.89.193:40909 (local address 127.0.0.1:59728)
0809 15:36:36.034462 (+ 268us) negotiation.cc:107] Waiting for socket to connect
0809 15:36:36.034476 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0809 15:36:36.034565 (+ 89us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 15:36:36.035343 (+ 778us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 15:36:36.035349 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 15:36:36.035386 (+ 37us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 15:36:36.035584 (+ 198us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:36:36.035594 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:36:36.037017 (+ 1423us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:36:36.037019 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:36:36.038571 (+ 1552us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:36:36.038582 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:36:36.039649 (+ 1067us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:36:36.039656 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:36:36.039735 (+ 79us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:36:36.039784 (+ 49us) client_negotiation.cc:769] Sending connection context
0809 15:36:36.039838 (+ 54us) client_negotiation.cc:240] Negotiation successful
0809 15:36:36.040378 (+ 540us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":210,"mutex_wait_us":611,"spinlock_wait_cycles":15360,"thread_start_us":100,"threads_started":1}
I20260809 15:36:36.041066 14968 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:36:36.033692 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.89.195:38177 (local address 127.0.0.1:43862)
0809 15:36:36.034077 (+ 385us) negotiation.cc:107] Waiting for socket to connect
0809 15:36:36.034121 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0809 15:36:36.034232 (+ 111us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 15:36:36.035364 (+ 1132us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 15:36:36.035368 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 15:36:36.035388 (+ 20us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 15:36:36.035764 (+ 376us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:36:36.035772 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:36:36.036971 (+ 1199us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:36:36.036975 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:36:36.038570 (+ 1595us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:36:36.038582 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:36:36.040092 (+ 1510us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:36:36.040095 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:36:36.040746 (+ 651us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:36:36.040768 (+ 22us) client_negotiation.cc:769] Sending connection context
0809 15:36:36.040856 (+ 88us) client_negotiation.cc:240] Negotiation successful
0809 15:36:36.040928 (+ 72us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":314,"mutex_wait_us":185,"spinlock_wait_cycles":6016,"thread_start_us":127,"threads_started":1}
I20260809 15:36:36.041612 14971 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:36:36.034611 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43862 (local address 127.3.89.195:38177)
0809 15:36:36.034948 (+ 337us) server_negotiation.cc:206] Beginning negotiation
0809 15:36:36.034982 (+ 34us) server_negotiation.cc:399] Waiting for connection header
0809 15:36:36.035002 (+ 20us) server_negotiation.cc:407] Connection header received
0809 15:36:36.035064 (+ 62us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 15:36:36.035069 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 15:36:36.035214 (+ 145us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 15:36:36.035892 (+ 678us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 15:36:36.036064 (+ 172us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:36:36.036831 (+ 767us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:36:36.039102 (+ 2271us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:36:36.039801 (+ 699us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:36:36.039907 (+ 106us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:36:36.040033 (+ 126us) server_negotiation.cc:1035] Waiting for connection context
0809 15:36:36.041383 (+ 1350us) server_negotiation.cc:299] Negotiation successful
0809 15:36:36.041481 (+ 98us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":249,"thread_start_us":120,"threads_started":1}
I20260809 15:36:36.043251 14741 tablet_service.cc:1467] Processing CreateTablet for tablet f7156abd378e4c2697072b51c5161e0f (DEFAULT_TABLE table=test-table [id=a572285bdd7f40deb8ff5e927ce9923f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 15:36:36.043273 14601 tablet_service.cc:1467] Processing CreateTablet for tablet f7156abd378e4c2697072b51c5161e0f (DEFAULT_TABLE table=test-table [id=a572285bdd7f40deb8ff5e927ce9923f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 15:36:36.044269 14741 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f7156abd378e4c2697072b51c5161e0f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 15:36:36.044292 14601 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f7156abd378e4c2697072b51c5161e0f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 15:36:36.044092 14882 tablet_service.cc:1467] Processing CreateTablet for tablet f7156abd378e4c2697072b51c5161e0f (DEFAULT_TABLE table=test-table [id=a572285bdd7f40deb8ff5e927ce9923f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 15:36:36.044983 14882 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f7156abd378e4c2697072b51c5161e0f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 15:36:36.050557 14972 tablet_bootstrap.cc:492] T f7156abd378e4c2697072b51c5161e0f P f4f519de9f1c470993535452c3ba1f1b: Bootstrap starting.
I20260809 15:36:36.051957 14973 tablet_bootstrap.cc:492] T f7156abd378e4c2697072b51c5161e0f P 1690f2c02cb84e4caf44219967afd6ee: Bootstrap starting.
I20260809 15:36:36.053243 14972 tablet_bootstrap.cc:654] T f7156abd378e4c2697072b51c5161e0f P f4f519de9f1c470993535452c3ba1f1b: Neither blocks nor log segments found. Creating new log.
I20260809 15:36:36.053659 14973 tablet_bootstrap.cc:654] T f7156abd378e4c2697072b51c5161e0f P 1690f2c02cb84e4caf44219967afd6ee: Neither blocks nor log segments found. Creating new log.
I20260809 15:36:36.053661 14974 tablet_bootstrap.cc:492] T f7156abd378e4c2697072b51c5161e0f P 074f582a381c46bb9cdaab6945c79e4e: Bootstrap starting.
I20260809 15:36:36.054023 14972 log.cc:824] T f7156abd378e4c2697072b51c5161e0f P f4f519de9f1c470993535452c3ba1f1b: Log is configured to *not* fsync() on all Append() calls
I20260809 15:36:36.054979 14973 log.cc:824] T f7156abd378e4c2697072b51c5161e0f P 1690f2c02cb84e4caf44219967afd6ee: Log is configured to *not* fsync() on all Append() calls
I20260809 15:36:36.055680 14974 tablet_bootstrap.cc:654] T f7156abd378e4c2697072b51c5161e0f P 074f582a381c46bb9cdaab6945c79e4e: Neither blocks nor log segments found. Creating new log.
I20260809 15:36:36.056524 14972 tablet_bootstrap.cc:492] T f7156abd378e4c2697072b51c5161e0f P f4f519de9f1c470993535452c3ba1f1b: No bootstrap required, opened a new log
I20260809 15:36:36.056736 14972 ts_tablet_manager.cc:1397] T f7156abd378e4c2697072b51c5161e0f P f4f519de9f1c470993535452c3ba1f1b: Time spent bootstrapping tablet: real 0.006s user 0.004s sys 0.000s
I20260809 15:36:36.056908 14974 log.cc:824] T f7156abd378e4c2697072b51c5161e0f P 074f582a381c46bb9cdaab6945c79e4e: Log is configured to *not* fsync() on all Append() calls
I20260809 15:36:36.057008 14973 tablet_bootstrap.cc:492] T f7156abd378e4c2697072b51c5161e0f P 1690f2c02cb84e4caf44219967afd6ee: No bootstrap required, opened a new log
I20260809 15:36:36.057180 14973 ts_tablet_manager.cc:1397] T f7156abd378e4c2697072b51c5161e0f P 1690f2c02cb84e4caf44219967afd6ee: Time spent bootstrapping tablet: real 0.005s user 0.004s sys 0.000s
I20260809 15:36:36.058801 14974 tablet_bootstrap.cc:492] T f7156abd378e4c2697072b51c5161e0f P 074f582a381c46bb9cdaab6945c79e4e: No bootstrap required, opened a new log
I20260809 15:36:36.059006 14974 ts_tablet_manager.cc:1397] T f7156abd378e4c2697072b51c5161e0f P 074f582a381c46bb9cdaab6945c79e4e: Time spent bootstrapping tablet: real 0.006s user 0.005s sys 0.000s
I20260809 15:36:36.059947 14972 raft_consensus.cc:348] T f7156abd378e4c2697072b51c5161e0f P f4f519de9f1c470993535452c3ba1f1b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f4f519de9f1c470993535452c3ba1f1b" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 44679 } } peers { permanent_uuid: "074f582a381c46bb9cdaab6945c79e4e" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 38177 } } peers { permanent_uuid: "1690f2c02cb84e4caf44219967afd6ee" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 40909 } }
I20260809 15:36:36.060164 14972 raft_consensus.cc:374] T f7156abd378e4c2697072b51c5161e0f P f4f519de9f1c470993535452c3ba1f1b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 15:36:36.060223 14972 raft_consensus.cc:729] T f7156abd378e4c2697072b51c5161e0f P f4f519de9f1c470993535452c3ba1f1b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f4f519de9f1c470993535452c3ba1f1b, State: Initialized, Role: FOLLOWER
I20260809 15:36:36.060643 14972 consensus_queue.cc:260] T f7156abd378e4c2697072b51c5161e0f P f4f519de9f1c470993535452c3ba1f1b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f4f519de9f1c470993535452c3ba1f1b" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 44679 } } peers { permanent_uuid: "074f582a381c46bb9cdaab6945c79e4e" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 38177 } } peers { permanent_uuid: "1690f2c02cb84e4caf44219967afd6ee" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 40909 } }
I20260809 15:36:36.060832 14973 raft_consensus.cc:348] T f7156abd378e4c2697072b51c5161e0f P 1690f2c02cb84e4caf44219967afd6ee [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f4f519de9f1c470993535452c3ba1f1b" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 44679 } } peers { permanent_uuid: "074f582a381c46bb9cdaab6945c79e4e" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 38177 } } peers { permanent_uuid: "1690f2c02cb84e4caf44219967afd6ee" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 40909 } }
I20260809 15:36:36.061066 14973 raft_consensus.cc:374] T f7156abd378e4c2697072b51c5161e0f P 1690f2c02cb84e4caf44219967afd6ee [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 15:36:36.061120 14973 raft_consensus.cc:729] T f7156abd378e4c2697072b51c5161e0f P 1690f2c02cb84e4caf44219967afd6ee [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1690f2c02cb84e4caf44219967afd6ee, State: Initialized, Role: FOLLOWER
I20260809 15:36:36.061362 14812 heartbeater.cc:503] Master 127.3.89.254:38337 was elected leader, sending a full tablet report...
I20260809 15:36:36.061573 14972 ts_tablet_manager.cc:1428] T f7156abd378e4c2697072b51c5161e0f P f4f519de9f1c470993535452c3ba1f1b: Time spent starting tablet: real 0.005s user 0.005s sys 0.000s
I20260809 15:36:36.061565 14973 consensus_queue.cc:260] T f7156abd378e4c2697072b51c5161e0f P 1690f2c02cb84e4caf44219967afd6ee [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f4f519de9f1c470993535452c3ba1f1b" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 44679 } } peers { permanent_uuid: "074f582a381c46bb9cdaab6945c79e4e" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 38177 } } peers { permanent_uuid: "1690f2c02cb84e4caf44219967afd6ee" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 40909 } }
I20260809 15:36:36.062474 14973 ts_tablet_manager.cc:1428] T f7156abd378e4c2697072b51c5161e0f P 1690f2c02cb84e4caf44219967afd6ee: Time spent starting tablet: real 0.005s user 0.006s sys 0.000s
I20260809 15:36:36.062794 14671 heartbeater.cc:503] Master 127.3.89.254:38337 was elected leader, sending a full tablet report...
I20260809 15:36:36.062620 14974 raft_consensus.cc:348] T f7156abd378e4c2697072b51c5161e0f P 074f582a381c46bb9cdaab6945c79e4e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f4f519de9f1c470993535452c3ba1f1b" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 44679 } } peers { permanent_uuid: "074f582a381c46bb9cdaab6945c79e4e" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 38177 } } peers { permanent_uuid: "1690f2c02cb84e4caf44219967afd6ee" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 40909 } }
I20260809 15:36:36.063728 14974 raft_consensus.cc:374] T f7156abd378e4c2697072b51c5161e0f P 074f582a381c46bb9cdaab6945c79e4e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 15:36:36.063781 14974 raft_consensus.cc:729] T f7156abd378e4c2697072b51c5161e0f P 074f582a381c46bb9cdaab6945c79e4e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 074f582a381c46bb9cdaab6945c79e4e, State: Initialized, Role: FOLLOWER
I20260809 15:36:36.064172 14974 consensus_queue.cc:260] T f7156abd378e4c2697072b51c5161e0f P 074f582a381c46bb9cdaab6945c79e4e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f4f519de9f1c470993535452c3ba1f1b" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 44679 } } peers { permanent_uuid: "074f582a381c46bb9cdaab6945c79e4e" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 38177 } } peers { permanent_uuid: "1690f2c02cb84e4caf44219967afd6ee" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 40909 } }
I20260809 15:36:36.064764 14955 heartbeater.cc:503] Master 127.3.89.254:38337 was elected leader, sending a full tablet report...
I20260809 15:36:36.065064 14974 ts_tablet_manager.cc:1428] T f7156abd378e4c2697072b51c5161e0f P 074f582a381c46bb9cdaab6945c79e4e: Time spent starting tablet: real 0.006s user 0.005s sys 0.000s
W20260809 15:36:36.067057 14673 tablet.cc:2369] T f7156abd378e4c2697072b51c5161e0f P 1690f2c02cb84e4caf44219967afd6ee: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260809 15:36:36.238528 14956 tablet.cc:2369] T f7156abd378e4c2697072b51c5161e0f P 074f582a381c46bb9cdaab6945c79e4e: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260809 15:36:36.271520 14813 tablet.cc:2369] T f7156abd378e4c2697072b51c5161e0f P f4f519de9f1c470993535452c3ba1f1b: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260809 15:36:36.290325 14979 raft_consensus.cc:482] T f7156abd378e4c2697072b51c5161e0f P 1690f2c02cb84e4caf44219967afd6ee [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 15:36:36.290549 14979 raft_consensus.cc:504] T f7156abd378e4c2697072b51c5161e0f P 1690f2c02cb84e4caf44219967afd6ee [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f4f519de9f1c470993535452c3ba1f1b" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 44679 } } peers { permanent_uuid: "074f582a381c46bb9cdaab6945c79e4e" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 38177 } } peers { permanent_uuid: "1690f2c02cb84e4caf44219967afd6ee" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 40909 } }
I20260809 15:36:36.291708 14979 leader_election.cc:288] T f7156abd378e4c2697072b51c5161e0f P 1690f2c02cb84e4caf44219967afd6ee [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers f4f519de9f1c470993535452c3ba1f1b (127.3.89.194:44679), 074f582a381c46bb9cdaab6945c79e4e (127.3.89.195:38177)
I20260809 15:36:36.297242 14971 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:36:36.292150 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.89.193:59103 (local address 127.3.89.195:38177)
0809 15:36:36.292271 (+ 121us) server_negotiation.cc:206] Beginning negotiation
0809 15:36:36.292275 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 15:36:36.292463 (+ 188us) server_negotiation.cc:407] Connection header received
0809 15:36:36.292612 (+ 149us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 15:36:36.292615 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 15:36:36.292663 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 15:36:36.292741 (+ 78us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 15:36:36.293217 (+ 476us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:36:36.294004 (+ 787us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:36:36.296002 (+ 1998us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:36:36.296609 (+ 607us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:36:36.296736 (+ 127us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:36:36.296818 (+ 82us) server_negotiation.cc:1035] Waiting for connection context
0809 15:36:36.297042 (+ 224us) server_negotiation.cc:299] Negotiation successful
0809 15:36:36.297111 (+ 69us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":43}
I20260809 15:36:36.297242 14967 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:36:36.291999 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.89.193:56569 (local address 127.3.89.194:44679)
0809 15:36:36.292126 (+ 127us) server_negotiation.cc:206] Beginning negotiation
0809 15:36:36.292130 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 15:36:36.292413 (+ 283us) server_negotiation.cc:407] Connection header received
0809 15:36:36.292586 (+ 173us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 15:36:36.292590 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 15:36:36.292645 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 15:36:36.292740 (+ 95us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 15:36:36.293217 (+ 477us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:36:36.294002 (+ 785us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:36:36.296002 (+ 2000us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:36:36.296613 (+ 611us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:36:36.296736 (+ 123us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:36:36.296786 (+ 50us) server_negotiation.cc:1035] Waiting for connection context
0809 15:36:36.297042 (+ 256us) server_negotiation.cc:299] Negotiation successful
0809 15:36:36.297112 (+ 70us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":51}
I20260809 15:36:36.297297 14982 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:36:36.291924 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.89.194:44679 (local address 127.3.89.193:56569)
0809 15:36:36.292302 (+ 378us) negotiation.cc:107] Waiting for socket to connect
0809 15:36:36.292320 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0809 15:36:36.292448 (+ 128us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 15:36:36.292768 (+ 320us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 15:36:36.292771 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 15:36:36.292811 (+ 40us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 15:36:36.293094 (+ 283us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:36:36.293102 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:36:36.294120 (+ 1018us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:36:36.294124 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:36:36.295860 (+ 1736us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:36:36.295870 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:36:36.296738 (+ 868us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:36:36.296742 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:36:36.296879 (+ 137us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:36:36.296927 (+ 48us) client_negotiation.cc:769] Sending connection context
0809 15:36:36.297007 (+ 80us) client_negotiation.cc:240] Negotiation successful
0809 15:36:36.297118 (+ 111us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":281,"mutex_wait_us":356,"spinlock_wait_cycles":26752,"thread_start_us":101,"threads_started":1}
I20260809 15:36:36.297299 14981 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:36:36.291955 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.89.195:38177 (local address 127.3.89.193:59103)
0809 15:36:36.292400 (+ 445us) negotiation.cc:107] Waiting for socket to connect
0809 15:36:36.292415 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0809 15:36:36.292506 (+ 91us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 15:36:36.292768 (+ 262us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 15:36:36.292771 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 15:36:36.292813 (+ 42us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 15:36:36.293097 (+ 284us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:36:36.293102 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:36:36.294120 (+ 1018us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:36:36.294124 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:36:36.295858 (+ 1734us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:36:36.295870 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:36:36.296738 (+ 868us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:36:36.296742 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:36:36.296876 (+ 134us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:36:36.296921 (+ 45us) client_negotiation.cc:769] Sending connection context
0809 15:36:36.297005 (+ 84us) client_negotiation.cc:240] Negotiation successful
0809 15:36:36.297122 (+ 117us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":383,"mutex_wait_us":298,"spinlock_wait_cycles":43648,"thread_start_us":110,"threads_started":1}
I20260809 15:36:36.297906 14761 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f7156abd378e4c2697072b51c5161e0f" candidate_uuid: "1690f2c02cb84e4caf44219967afd6ee" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f4f519de9f1c470993535452c3ba1f1b" is_pre_election: true
I20260809 15:36:36.298050 14903 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f7156abd378e4c2697072b51c5161e0f" candidate_uuid: "1690f2c02cb84e4caf44219967afd6ee" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "074f582a381c46bb9cdaab6945c79e4e" is_pre_election: true
I20260809 15:36:36.298205 14761 raft_consensus.cc:2454] T f7156abd378e4c2697072b51c5161e0f P f4f519de9f1c470993535452c3ba1f1b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1690f2c02cb84e4caf44219967afd6ee in term 0.
I20260809 15:36:36.298382 14903 raft_consensus.cc:2454] T f7156abd378e4c2697072b51c5161e0f P 074f582a381c46bb9cdaab6945c79e4e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1690f2c02cb84e4caf44219967afd6ee in term 0.
I20260809 15:36:36.298740 14553 leader_election.cc:302] T f7156abd378e4c2697072b51c5161e0f P 1690f2c02cb84e4caf44219967afd6ee [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1690f2c02cb84e4caf44219967afd6ee, f4f519de9f1c470993535452c3ba1f1b; no voters:
I20260809 15:36:36.299019 14979 raft_consensus.cc:2781] T f7156abd378e4c2697072b51c5161e0f P 1690f2c02cb84e4caf44219967afd6ee [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260809 15:36:36.299120 14979 raft_consensus.cc:482] T f7156abd378e4c2697072b51c5161e0f P 1690f2c02cb84e4caf44219967afd6ee [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 15:36:36.299154 14979 raft_consensus.cc:3037] T f7156abd378e4c2697072b51c5161e0f P 1690f2c02cb84e4caf44219967afd6ee [term 0 FOLLOWER]: Advancing to term 1
I20260809 15:36:36.300422 14979 raft_consensus.cc:504] T f7156abd378e4c2697072b51c5161e0f P 1690f2c02cb84e4caf44219967afd6ee [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f4f519de9f1c470993535452c3ba1f1b" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 44679 } } peers { permanent_uuid: "074f582a381c46bb9cdaab6945c79e4e" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 38177 } } peers { permanent_uuid: "1690f2c02cb84e4caf44219967afd6ee" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 40909 } }
I20260809 15:36:36.300792 14979 leader_election.cc:288] T f7156abd378e4c2697072b51c5161e0f P 1690f2c02cb84e4caf44219967afd6ee [CANDIDATE]: Term 1 election: Requested vote from peers f4f519de9f1c470993535452c3ba1f1b (127.3.89.194:44679), 074f582a381c46bb9cdaab6945c79e4e (127.3.89.195:38177)
I20260809 15:36:36.301110 14761 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f7156abd378e4c2697072b51c5161e0f" candidate_uuid: "1690f2c02cb84e4caf44219967afd6ee" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f4f519de9f1c470993535452c3ba1f1b"
I20260809 15:36:36.301172 14903 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f7156abd378e4c2697072b51c5161e0f" candidate_uuid: "1690f2c02cb84e4caf44219967afd6ee" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "074f582a381c46bb9cdaab6945c79e4e"
I20260809 15:36:36.301239 14761 raft_consensus.cc:3037] T f7156abd378e4c2697072b51c5161e0f P f4f519de9f1c470993535452c3ba1f1b [term 0 FOLLOWER]: Advancing to term 1
I20260809 15:36:36.301276 14903 raft_consensus.cc:3037] T f7156abd378e4c2697072b51c5161e0f P 074f582a381c46bb9cdaab6945c79e4e [term 0 FOLLOWER]: Advancing to term 1
I20260809 15:36:36.302517 14761 raft_consensus.cc:2454] T f7156abd378e4c2697072b51c5161e0f P f4f519de9f1c470993535452c3ba1f1b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1690f2c02cb84e4caf44219967afd6ee in term 1.
I20260809 15:36:36.302517 14903 raft_consensus.cc:2454] T f7156abd378e4c2697072b51c5161e0f P 074f582a381c46bb9cdaab6945c79e4e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1690f2c02cb84e4caf44219967afd6ee in term 1.
I20260809 15:36:36.302816 14553 leader_election.cc:302] T f7156abd378e4c2697072b51c5161e0f P 1690f2c02cb84e4caf44219967afd6ee [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1690f2c02cb84e4caf44219967afd6ee, f4f519de9f1c470993535452c3ba1f1b; no voters:
I20260809 15:36:36.303020 14979 raft_consensus.cc:2781] T f7156abd378e4c2697072b51c5161e0f P 1690f2c02cb84e4caf44219967afd6ee [term 1 FOLLOWER]: Leader election won for term 1
I20260809 15:36:36.303247 14979 raft_consensus.cc:686] T f7156abd378e4c2697072b51c5161e0f P 1690f2c02cb84e4caf44219967afd6ee [term 1 LEADER]: Becoming Leader. State: Replica: 1690f2c02cb84e4caf44219967afd6ee, State: Running, Role: LEADER
I20260809 15:36:36.303580 14979 consensus_queue.cc:237] T f7156abd378e4c2697072b51c5161e0f P 1690f2c02cb84e4caf44219967afd6ee [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f4f519de9f1c470993535452c3ba1f1b" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 44679 } } peers { permanent_uuid: "074f582a381c46bb9cdaab6945c79e4e" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 38177 } } peers { permanent_uuid: "1690f2c02cb84e4caf44219967afd6ee" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 40909 } }
I20260809 15:36:36.306617 14467 catalog_manager.cc:5495] T f7156abd378e4c2697072b51c5161e0f P 1690f2c02cb84e4caf44219967afd6ee reported cstate change: term changed from 0 to 1, leader changed from <none> to 1690f2c02cb84e4caf44219967afd6ee (127.3.89.193). New cstate: current_term: 1 leader_uuid: "1690f2c02cb84e4caf44219967afd6ee" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f4f519de9f1c470993535452c3ba1f1b" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 44679 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "074f582a381c46bb9cdaab6945c79e4e" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 38177 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "1690f2c02cb84e4caf44219967afd6ee" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 40909 } health_report { overall_health: HEALTHY } } }
I20260809 15:36:36.328780 14970 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:36:36.325653 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59740 (local address 127.3.89.193:40909)
0809 15:36:36.325774 (+ 121us) server_negotiation.cc:206] Beginning negotiation
0809 15:36:36.325777 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0809 15:36:36.325790 (+ 13us) server_negotiation.cc:407] Connection header received
0809 15:36:36.325879 (+ 89us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 15:36:36.325881 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 15:36:36.325929 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 15:36:36.326014 (+ 85us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 15:36:36.326218 (+ 204us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:36:36.326709 (+ 491us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:36:36.327533 (+ 824us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:36:36.327736 (+ 203us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:36:36.327813 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:36:36.328070 (+ 257us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 15:36:36.328422 (+ 352us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 15:36:36.328518 (+ 96us) server_negotiation.cc:1035] Waiting for connection context
0809 15:36:36.328572 (+ 54us) server_negotiation.cc:299] Negotiation successful
0809 15:36:36.328619 (+ 47us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":48}
I20260809 15:36:36.334394 14903 raft_consensus.cc:1261] T f7156abd378e4c2697072b51c5161e0f P 074f582a381c46bb9cdaab6945c79e4e [term 1 FOLLOWER]: Refusing update from remote peer 1690f2c02cb84e4caf44219967afd6ee: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 15:36:36.334394 14761 raft_consensus.cc:1261] T f7156abd378e4c2697072b51c5161e0f P f4f519de9f1c470993535452c3ba1f1b [term 1 FOLLOWER]: Refusing update from remote peer 1690f2c02cb84e4caf44219967afd6ee: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 15:36:36.335229 14979 consensus_queue.cc:1035] T f7156abd378e4c2697072b51c5161e0f P 1690f2c02cb84e4caf44219967afd6ee [LEADER]: Connected to new peer: Peer: permanent_uuid: "f4f519de9f1c470993535452c3ba1f1b" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 44679 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 15:36:36.335444 14983 consensus_queue.cc:1035] T f7156abd378e4c2697072b51c5161e0f P 1690f2c02cb84e4caf44219967afd6ee [LEADER]: Connected to new peer: Peer: permanent_uuid: "074f582a381c46bb9cdaab6945c79e4e" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 38177 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 15:36:36.342147 14988 mvcc.cc:205] Tried to move back new op lower bound from 7316643005778407424 to 7316643005662175232. Current Snapshot: MvccSnapshot[applied={T|T < 7316643005778407424}]
I20260809 15:36:36.343099 14990 mvcc.cc:205] Tried to move back new op lower bound from 7316643005778407424 to 7316643005662175232. Current Snapshot: MvccSnapshot[applied={T|T < 7316643005778407424}]
I20260809 15:36:36.344700 14989 mvcc.cc:205] Tried to move back new op lower bound from 7316643005778407424 to 7316643005662175232. Current Snapshot: MvccSnapshot[applied={T|T < 7316643005778407424}]
I20260809 15:36:36.350217 14467 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:36634:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 15:36:36.350343 14467 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:36634:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 15:36:36.352675 14467 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 74b4eb3f44224d5582d2c1cfaaf0da55: Sending DeleteTablet for 3 replicas of tablet f7156abd378e4c2697072b51c5161e0f
I20260809 15:36:36.353397 14741 tablet_service.cc:1514] Processing DeleteTablet for tablet f7156abd378e4c2697072b51c5161e0f with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 15:36:36 UTC) from {username='kudu', principal='kudu/127.3.89.254@KRBTEST.COM'} at 127.0.0.1:52952
I20260809 15:36:36.353454 14601 tablet_service.cc:1514] Processing DeleteTablet for tablet f7156abd378e4c2697072b51c5161e0f with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 15:36:36 UTC) from {username='kudu', principal='kudu/127.3.89.254@KRBTEST.COM'} at 127.0.0.1:59728
I20260809 15:36:36.353755 14882 tablet_service.cc:1514] Processing DeleteTablet for tablet f7156abd378e4c2697072b51c5161e0f with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 15:36:36 UTC) from {username='kudu', principal='kudu/127.3.89.254@KRBTEST.COM'} at 127.0.0.1:43862
I20260809 15:36:36.354022 14997 tablet_replica.cc:330] T f7156abd378e4c2697072b51c5161e0f P 1690f2c02cb84e4caf44219967afd6ee: stopping tablet replica
I20260809 15:36:36.354136 14998 tablet_replica.cc:330] T f7156abd378e4c2697072b51c5161e0f P f4f519de9f1c470993535452c3ba1f1b: stopping tablet replica
I20260809 15:36:36.354161 3431 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskm314YR/build/debug/bin/kudu with pid 14534
I20260809 15:36:36.354422 14998 raft_consensus.cc:2229] T f7156abd378e4c2697072b51c5161e0f P f4f519de9f1c470993535452c3ba1f1b [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 15:36:36.354709 14998 raft_consensus.cc:2258] T f7156abd378e4c2697072b51c5161e0f P f4f519de9f1c470993535452c3ba1f1b [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 15:36:36.354867 14999 tablet_replica.cc:330] T f7156abd378e4c2697072b51c5161e0f P 074f582a381c46bb9cdaab6945c79e4e: stopping tablet replica
I20260809 15:36:36.355141 14999 raft_consensus.cc:2229] T f7156abd378e4c2697072b51c5161e0f P 074f582a381c46bb9cdaab6945c79e4e [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 15:36:36.355381 14999 raft_consensus.cc:2258] T f7156abd378e4c2697072b51c5161e0f P 074f582a381c46bb9cdaab6945c79e4e [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 15:36:36.356536 14998 ts_tablet_manager.cc:1905] T f7156abd378e4c2697072b51c5161e0f P f4f519de9f1c470993535452c3ba1f1b: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 15:36:36.358947 14999 ts_tablet_manager.cc:1905] T f7156abd378e4c2697072b51c5161e0f P 074f582a381c46bb9cdaab6945c79e4e: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 15:36:36.359586 14998 ts_tablet_manager.cc:1918] T f7156abd378e4c2697072b51c5161e0f P f4f519de9f1c470993535452c3ba1f1b: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 15:36:36.359684 14998 log.cc:1196] T f7156abd378e4c2697072b51c5161e0f P f4f519de9f1c470993535452c3ba1f1b: Deleting WAL directory at /tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786289732823074-3431-0/minicluster-data/ts-1/wal/wals/f7156abd378e4c2697072b51c5161e0f
I20260809 15:36:36.360009 14998 ts_tablet_manager.cc:1939] T f7156abd378e4c2697072b51c5161e0f P f4f519de9f1c470993535452c3ba1f1b: Deleting consensus metadata
I20260809 15:36:36.361189 14454 catalog_manager.cc:4841] TS f4f519de9f1c470993535452c3ba1f1b (127.3.89.194:44679): tablet f7156abd378e4c2697072b51c5161e0f (table test-table [id=a572285bdd7f40deb8ff5e927ce9923f]) successfully deleted
I20260809 15:36:36.361714 14999 ts_tablet_manager.cc:1918] T f7156abd378e4c2697072b51c5161e0f P 074f582a381c46bb9cdaab6945c79e4e: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 15:36:36.361791 14999 log.cc:1196] T f7156abd378e4c2697072b51c5161e0f P 074f582a381c46bb9cdaab6945c79e4e: Deleting WAL directory at /tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786289732823074-3431-0/minicluster-data/ts-2/wal/wals/f7156abd378e4c2697072b51c5161e0f
I20260809 15:36:36.362100 14999 ts_tablet_manager.cc:1939] T f7156abd378e4c2697072b51c5161e0f P 074f582a381c46bb9cdaab6945c79e4e: Deleting consensus metadata
I20260809 15:36:36.363080 14453 catalog_manager.cc:4841] TS 074f582a381c46bb9cdaab6945c79e4e (127.3.89.195:38177): tablet f7156abd378e4c2697072b51c5161e0f (table test-table [id=a572285bdd7f40deb8ff5e927ce9923f]) successfully deleted
W20260809 15:36:36.363791 14453 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.3.89.193:40909 (error 108)
I20260809 15:36:36.364032 3431 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskm314YR/build/debug/bin/kudu with pid 14677
I20260809 15:36:36.366411 14968 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:36:36.366085 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.89.193:40909 (local address 127.0.0.1:59756)
0809 15:36:36.366245 (+ 160us) negotiation.cc:107] Waiting for socket to connect
0809 15:36:36.366327 (+ 82us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.89.193:40909: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":54}
W20260809 15:36:36.371126 14453 catalog_manager.cc:4584] TS 1690f2c02cb84e4caf44219967afd6ee (127.3.89.193:40909): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet f7156abd378e4c2697072b51c5161e0f: Network error: Client connection negotiation failed: client connection to 127.3.89.193:40909: connect: Connection refused (error 111)
I20260809 15:36:36.371214 3431 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskm314YR/build/debug/bin/kudu with pid 14817
I20260809 15:36:36.377811 3431 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskm314YR/build/debug/bin/kudu with pid 14434
2026-08-09T15:36:36Z chronyd exiting
[ OK ] SecurityITest.TestRequireAuthenticationSecureCluster (2443 ms)
[ RUN ] SecurityITest.TestEncryptionWithKMSIntegration
Loading random data
Initializing database '/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786289732823074-3431-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 15:36:36 dist-test-slave-2w3w krb5kdc[15003](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 15:36:36 dist-test-slave-2w3w krb5kdc[15003](info): set up 2 sockets
Aug 09 15:36:36 dist-test-slave-2w3w krb5kdc[15003](info): commencing operation
krb5kdc: starting...
W20260809 15:36:37.457896 3431 mini_kdc.cc:121] Time spent starting KDC: real 1.056s user 0.000s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786289732823074-3431-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786289732823074-3431-0/krb5kdc/test-user.keytab.
Aug 09 15:36:37 dist-test-slave-2w3w krb5kdc[15003](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786289797, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T15:36:37Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T15:36:37Z Disabled control of system clock
WARNING: no policy specified for rangeradmin/127.3.89.212@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangeradmin/127.3.89.212@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangeradmin/127.3.89.212@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786289732823074-3431-0/krb5kdc/rangeradmin_127.3.89.212@KRBTEST.COM.keytab.
Entry for principal rangeradmin/127.3.89.212@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786289732823074-3431-0/krb5kdc/rangeradmin_127.3.89.212@KRBTEST.COM.keytab.
WARNING: no policy specified for rangerlookup/127.3.89.212@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangerlookup/127.3.89.212@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangerlookup/127.3.89.212@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786289732823074-3431-0/krb5kdc/rangerlookup_127.3.89.212@KRBTEST.COM.keytab.
Entry for principal rangerlookup/127.3.89.212@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786289732823074-3431-0/krb5kdc/rangerlookup_127.3.89.212@KRBTEST.COM.keytab.
WARNING: no policy specified for HTTP/127.3.89.212@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "HTTP/127.3.89.212@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal HTTP/127.3.89.212@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786289732823074-3431-0/krb5kdc/HTTP_127.3.89.212@KRBTEST.COM.keytab.
Entry for principal HTTP/127.3.89.212@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786289732823074-3431-0/krb5kdc/HTTP_127.3.89.212@KRBTEST.COM.keytab.
I20260809 15:36:37.625837 3431 mini_postgres.cc:62] Running initdb...
The files belonging to this database system will be owned by user "slave".
This user must also own the server process.
The database cluster will be initialized with locale "C".
The default database encoding has accordingly been set to "SQL_ASCII".
The default text search configuration will be set to "english".
Data page checksums are disabled.
creating directory /tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786289732823074-3431-0/minicluster-data/postgres ... ok
creating subdirectories ... ok
selecting dynamic shared memory implementation ... posix
selecting default max_connections ... 100
selecting default shared_buffers ... 128MB
selecting default time zone ... Etc/UTC
creating configuration files ... ok
running bootstrap script ... ok
performing post-bootstrap initialization ... ok
syncing data to disk ... ok
initdb: warning: enabling "trust" authentication for local connections
You can change this by editing pg_hba.conf or using the option -A, or
--auth-local and --auth-host, the next time you run initdb.
Success. You can now start the database server using:
/tmp/dist-test-taskm314YR/build/debug/bin/postgres/pg_ctl -D /tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786289732823074-3431-0/minicluster-data/postgres -l logfile start
2026-08-09 15:36:40.066 UTC [15035] LOG: starting PostgreSQL 12.2 on x86_64-pc-linux-gnu, compiled by cc (GCC) 8.3.1 20190311 (Red Hat 8.3.1-3), 64-bit
2026-08-09 15:36:40.066 UTC [15035] LOG: listening on IPv4 address "127.3.89.212", port 34095
2026-08-09 15:36:40.069 UTC [15035] LOG: listening on Unix socket "/tmp/.s.PGSQL.34095"
2026-08-09 15:36:40.090 UTC [15038] LOG: database system was shut down at 2026-08-09 15:36:38 UTC
2026-08-09 15:36:40.094 UTC [15035] LOG: database system is ready to accept connections
I20260809 15:36:41.088073 3431 mini_postgres.cc:96] Postgres bound to 34095
2026-08-09 15:36:41.095 UTC [15046] FATAL: database "slave" does not exist
127.3.89.212:34095 - accepting connections
I20260809 15:36:41.096752 3431 mini_ranger.cc:161] Starting Ranger...
I20260809 15:36:41.112993 3431 mini_ranger.cc:84] Created miniranger Postgres user
I20260809 15:36:41.580842 3431 mini_ranger.cc:87] Created ranger Postgres database
I20260809 15:36:41.580927 3431 mini_ranger.cc:178] Starting Ranger out of /tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786289732823074-3431-0/minicluster-data/ranger-admin
2026-08-09 15:36:41,871 [I] DB FLAVOR :POSTGRES
2026-08-09 15:36:41,871 [I] --------- Verifying Ranger DB connection ---------
2026-08-09 15:36:41,871 [I] Checking connection..
2026-08-09 15:36:41,872 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskm314YR/build/debug/bin/postgresql.jar:/tmp/dist-test-taskm314YR/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.3.89.212:34095/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select 1;"
2026-08-09 15:36:42,137 [I] Checking connection passed.
2026-08-09 15:36:42,137 [I] --------- Verifying version history table ---------
2026-08-09 15:36:42,137 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskm314YR/build/debug/bin/postgresql.jar:/tmp/dist-test-taskm314YR/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.3.89.212:34095/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-08-09 15:36:42,398 [I] Table x_db_version_h does not exist in database ranger
2026-08-09 15:36:42,398 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskm314YR/build/debug/bin/postgresql.jar:/tmp/dist-test-taskm314YR/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.3.89.212:34095/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-08-09 15:36:42,640 [I] Table x_db_version_h does not exist in database ranger
2026-08-09 15:36:42,640 [I] Importing x_db_version_h table schema to database ranger from file: create_dbversion_catalog.sql
2026-08-09 15:36:42,640 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskm314YR/build/debug/bin/postgresql.jar:/tmp/dist-test-taskm314YR/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.3.89.212:34095/ranger -u miniranger -p '********' -noheader -trim -c \; -input /tmp/dist-test-taskm314YR/thirdparty/src/ranger-2.1.0-admin/db/postgres/create_dbversion_catalog.sql
2026-08-09 15:36:42.891 UTC [15115] WARNING: there is no transaction in progress
2026-08-09 15:36:42,898 [I] create_dbversion_catalog.sql file imported successfully
2026-08-09 15:36:42,899 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskm314YR/build/debug/bin/postgresql.jar:/tmp/dist-test-taskm314YR/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.3.89.212:34095/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-08-09 15:36:43,161 [I] Table x_db_version_h already exists in database 'ranger'
2026-08-09 15:36:43,161 [I] --------- Importing Ranger Core DB Schema ---------
2026-08-09 15:36:43,162 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskm314YR/build/debug/bin/postgresql.jar:/tmp/dist-test-taskm314YR/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.3.89.212:34095/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'CORE_DB_SCHEMA' and active = 'Y';"
2026-08-09 15:36:43,421 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskm314YR/build/debug/bin/postgresql.jar:/tmp/dist-test-taskm314YR/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.3.89.212:34095/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'CORE_DB_SCHEMA' and active = 'N';"
2026-08-09 15:36:43,685 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskm314YR/build/debug/bin/postgresql.jar:/tmp/dist-test-taskm314YR/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.3.89.212:34095/ranger -u miniranger -p '********' -noheader -trim -c \; -query "insert into x_db_version_h (version, inst_at, inst_by, updated_at, updated_by,active) values ('CORE_DB_SCHEMA', current_timestamp, 'Ranger 2.1.0', current_timestamp, 'dist-test-slave-2w3w.c.gcp-upstream.internal','N') ;"
2026-08-09 15:36:43,928 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskm314YR/build/debug/bin/postgresql.jar:/tmp/dist-test-taskm314YR/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.3.89.212:34095/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_portal_user') as temp;"
2026-08-09 15:36:44,192 [I] Table x_portal_user does not exist in database ranger
2026-08-09 15:36:44,193 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskm314YR/build/debug/bin/postgresql.jar:/tmp/dist-test-taskm314YR/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.3.89.212:34095/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_policy_ref_group') as temp;"
2026-08-09 15:36:44,443 [I] Table x_policy_ref_group does not exist in database ranger
2026-08-09 15:36:44,443 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskm314YR/build/debug/bin/postgresql.jar:/tmp/dist-test-taskm314YR/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.3.89.212:34095/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and active = 'Y';"
2026-08-09 15:36:44,697 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskm314YR/build/debug/bin/postgresql.jar:/tmp/dist-test-taskm314YR/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.3.89.212:34095/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'JAVA_PATCHES' and active = 'Y';"
2026-08-09 15:36:44,959 [I] Importing DB schema to database ranger from file: ranger_core_db_postgres.sql
2026-08-09 15:36:44,960 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskm314YR/build/debug/bin/postgresql.jar:/tmp/dist-test-taskm314YR/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.3.89.212:34095/ranger -u miniranger -p '********' -noheader -trim -c \; -input /tmp/dist-test-taskm314YR/thirdparty/src/ranger-2.1.0-admin/db/postgres/optimized/current/ranger_core_db_postgres.sql
2026-08-09 15:36:45.901 UTC [15232] WARNING: there is no transaction in progress
2026-08-09 15:36:45.917 UTC [15232] WARNING: there is no transaction in progress
2026-08-09 15:36:45.933 UTC [15232] WARNING: there is no transaction in progress
2026-08-09 15:36:45.948 UTC [15232] WARNING: there is no transaction in progress
2026-08-09 15:36:45.967 UTC [15232] WARNING: there is no transaction in progress
2026-08-09 15:36:45.987 UTC [15232] WARNING: there is no transaction in progress
2026-08-09 15:36:46.043 UTC [15232] WARNING: there is no transaction in progress
2026-08-09 15:36:46.049 UTC [15232] WARNING: there is no transaction in progress
2026-08-09 15:36:46.064 UTC [15232] WARNING: there is no transaction in progress
2026-08-09 15:36:46.074 UTC [15232] WARNING: there is no transaction in progress
2026-08-09 15:36:46.084 UTC [15232] WARNING: there is no transaction in progress
2026-08-09 15:36:46.097 UTC [15232] WARNING: there is no transaction in progress
2026-08-09 15:36:46.107 UTC [15232] WARNING: there is no transaction in progress
2026-08-09 15:36:46.113 UTC [15232] WARNING: there is no transaction in progress
2026-08-09 15:36:46.120 UTC [15232] WARNING: there is no transaction in progress
2026-08-09 15:36:46.127 UTC [15232] WARNING: there is no transaction in progress
2026-08-09 15:36:46,745 [I] ranger_core_db_postgres.sql file imported successfully
2026-08-09 15:36:46,746 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskm314YR/build/debug/bin/postgresql.jar:/tmp/dist-test-taskm314YR/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.3.89.212:34095/ranger -u miniranger -p '********' -noheader -trim -c \; -query "update x_db_version_h set inst_by='Ranger 2.1.0' where active='Y' and updated_by='localhost';"
2026-08-09 15:36:47,003 [I] Patches status entries updated from base ranger version to current installed ranger version:Ranger 2.1.0
2026-08-09 15:36:47,003 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskm314YR/build/debug/bin/postgresql.jar:/tmp/dist-test-taskm314YR/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.3.89.212:34095/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_portal_user') as temp;"
2026-08-09 15:36:47,270 [I] Table x_portal_user already exists in database 'ranger'
2026-08-09 15:36:47,270 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskm314YR/build/debug/bin/postgresql.jar:/tmp/dist-test-taskm314YR/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.3.89.212:34095/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_policy_ref_group') as temp;"
2026-08-09 15:36:47,513 [I] Table x_policy_ref_group already exists in database 'ranger'
2026-08-09 15:36:47,514 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskm314YR/build/debug/bin/postgresql.jar:/tmp/dist-test-taskm314YR/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.3.89.212:34095/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and active = 'Y';"
2026-08-09 15:36:47,758 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskm314YR/build/debug/bin/postgresql.jar:/tmp/dist-test-taskm314YR/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.3.89.212:34095/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'JAVA_PATCHES' and active = 'Y';"
2026-08-09 15:36:48,019 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskm314YR/build/debug/bin/postgresql.jar:/tmp/dist-test-taskm314YR/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.3.89.212:34095/ranger -u miniranger -p '********' -noheader -trim -c \; -query "update x_db_version_h set active='Y' where version='CORE_DB_SCHEMA' and active='N' and updated_by='dist-test-slave-2w3w.c.gcp-upstream.internal';"
2026-08-09 15:36:48,267 [I] CORE_DB_SCHEMA import status has been updated
2026-08-09 15:36:48,267 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskm314YR/build/debug/bin/postgresql.jar:/tmp/dist-test-taskm314YR/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.3.89.212:34095/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and inst_by = 'Ranger 2.1.0' and active = 'Y';"
2026-08-09 15:36:48,533 [I] DB_PATCHES have already been applied
I20260809 15:36:48.539541 3431 mini_ranger.cc:191] Using Ranger class path: /tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786289732823074-3431-0/minicluster-data/ranger-admin:/tmp/dist-test-taskm314YR/thirdparty/src/ranger-2.1.0-admin/ews/lib/*:/usr/lib/jvm/java-1.8.0-openjdk-amd64/lib/*:/tmp/dist-test-taskm314YR/thirdparty/src/hadoop-3.2.0/*:/tmp/dist-test-taskm314YR/build/debug/bin/postgresql.jar:/tmp/dist-test-taskm314YR/thirdparty/src/ranger-2.1.0-admin/ews/webapp
I20260809 15:36:48.539603 3431 mini_ranger.cc:193] Using host: 127.3.89.212
I20260809 15:36:48.543859 3431 mini_ranger.cc:227] Ranger admin URL: http://127.3.89.212:47571
Aug 09, 2026 3:36:48 PM org.apache.ranger.server.tomcat.EmbeddedServer getKeyManagers
WARNING: Config 'ranger.keystore.file' or 'ranger.service.https.attrib.keystore.file' is not found or contains blank value
Aug 09, 2026 3:36:48 PM org.apache.ranger.server.tomcat.EmbeddedServer getTrustManagers
WARNING: Config 'ranger.truststore.file' is not found or contains blank value!
Aug 09, 2026 3:36:48 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Deriving webapp folder from catalina.base property. folder=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786289732823074-3431-0/minicluster-data/ranger-admin/ews/webapp
Aug 09, 2026 3:36:48 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Webapp file =/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786289732823074-3431-0/minicluster-data/ranger-admin/ews/webapp, webAppName = /
Aug 09, 2026 3:36:48 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Adding webapp [/] = path [/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786289732823074-3431-0/minicluster-data/ranger-admin/ews/webapp] .....
Aug 09, 2026 3:36:49 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Finished init of webapp [/] = path [/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786289732823074-3431-0/minicluster-data/ranger-admin/ews/webapp].
Aug 09, 2026 3:36:49 PM org.apache.ranger.server.tomcat.EmbeddedServer startServer
INFO: Server Name : miniranger
I20260809 15:36:49.689131 3431 mini_ranger.cc:160] Time spent starting Ranger: real 8.592s user 0.002s sys 0.011s
Aug 09, 2026 3:37:09 PM com.sun.jersey.api.core.PackagesResourceConfig init
INFO: Scanning for root resource and provider classes in the packages:
org.apache.ranger.rest
org.apache.ranger.common
xa.rest
Aug 09, 2026 3:37:09 PM com.sun.jersey.api.core.ScanningResourceConfig logClasses
INFO: Root resource classes found:
class org.apache.ranger.rest.MetricsREST
class org.apache.ranger.rest.XKeyREST
class org.apache.ranger.rest.XAuditREST
class org.apache.ranger.rest.XUserREST
class org.apache.ranger.rest.UserREST
class org.apache.ranger.rest.ServiceREST
class org.apache.ranger.rest.PublicAPIs
class org.apache.ranger.rest.TagREST
class org.apache.ranger.rest.AssetREST
class org.apache.ranger.rest.RoleREST
class org.apache.ranger.rest.SecurityZoneREST
class org.apache.ranger.rest.PublicAPIsv2
Aug 09, 2026 3:37:09 PM com.sun.jersey.api.core.ScanningResourceConfig logClasses
INFO: Provider classes found:
class org.apache.ranger.common.RangerJAXBContextResolver
Aug 09, 2026 3:37:10 PM com.sun.jersey.spi.spring.container.servlet.SpringServlet getContext
INFO: Using default applicationContext
Aug 09, 2026 3:37:10 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, assetREST, of type org.apache.ranger.rest.AssetREST as a root resource class
Aug 09, 2026 3:37:10 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, metricsREST, of type org.apache.ranger.rest.MetricsREST as a root resource class
Aug 09, 2026 3:37:10 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, publicAPIs, of type org.apache.ranger.rest.PublicAPIs as a root resource class
Aug 09, 2026 3:37:10 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, publicAPIsv2, of type org.apache.ranger.rest.PublicAPIsv2 as a root resource class
Aug 09, 2026 3:37:10 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, roleREST, of type org.apache.ranger.rest.RoleREST as a root resource class
Aug 09, 2026 3:37:10 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, securityZoneREST, of type org.apache.ranger.rest.SecurityZoneREST as a root resource class
Aug 09, 2026 3:37:10 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, serviceREST, of type org.apache.ranger.rest.ServiceREST as a root resource class
Aug 09, 2026 3:37:10 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, tagREST, of type org.apache.ranger.rest.TagREST as a root resource class
Aug 09, 2026 3:37:10 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, userREST, of type org.apache.ranger.rest.UserREST as a root resource class
Aug 09, 2026 3:37:10 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XAuditREST, of type org.apache.ranger.rest.XAuditREST as a root resource class
Aug 09, 2026 3:37:10 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XKeyREST, of type org.apache.ranger.rest.XKeyREST as a root resource class
Aug 09, 2026 3:37:10 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XUserREST, of type org.apache.ranger.rest.XUserREST as a root resource class
Aug 09, 2026 3:37:10 PM com.sun.jersey.server.impl.application.WebApplicationImpl _initiate
INFO: Initiating Jersey application, version 'Jersey: 1.19.3 10/24/2016 03:58 PM'
Aug 09, 2026 3:37:10 PM com.sun.jersey.spi.inject.Errors processErrorMessages
WARNING: The following warnings have been detected with resource and/or provider classes:
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInExcel(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse), MUST return a non-void type.
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInJson(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse,java.lang.Boolean), MUST return a non-void type.
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInCsv(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse) throws java.io.IOException, MUST return a non-void type.
I20260809 15:37:11.501639 3431 mini_ranger.cc:261] Created Kudu service
WARNING: no policy specified for rangerkms/127.3.89.212@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangerkms/127.3.89.212@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangerkms/127.3.89.212@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786289732823074-3431-0/krb5kdc/rangerkms_127.3.89.212@KRBTEST.COM.keytab.
Entry for principal rangerkms/127.3.89.212@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786289732823074-3431-0/krb5kdc/rangerkms_127.3.89.212@KRBTEST.COM.keytab.
WARNING: no policy specified for HTTP/127.3.89.212@KRBTEST.COM; defaulting to no policy
add_principal: Principal or policy already exists while creating "HTTP/127.3.89.212@KRBTEST.COM".
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal HTTP/127.3.89.212@KRBTEST.COM with kvno 3, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786289732823074-3431-0/krb5kdc/HTTP_127.3.89.212@KRBTEST.COM.keytab.
Entry for principal HTTP/127.3.89.212@KRBTEST.COM with kvno 3, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786289732823074-3431-0/krb5kdc/HTTP_127.3.89.212@KRBTEST.COM.keytab.
WARNING: no policy specified for keyadmin@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "keyadmin@KRBTEST.COM" created.
Aug 09 15:37:11 dist-test-slave-2w3w krb5kdc[15003](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786289831, etypes {rep=17 tkt=17 ses=17}, keyadmin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for keyadmin@KRBTEST.COM:
I20260809 15:37:11.589465 3431 mini_ranger_kms.cc:208] Starting Ranger KMS...
I20260809 15:37:11.600106 3431 mini_ranger_kms.cc:78] Created minirangerkms Postgres user
I20260809 15:37:12.332985 3431 mini_ranger_kms.cc:81] Created rangerkms Postgres database
I20260809 15:37:12.333091 3431 mini_ranger_kms.cc:226] Starting Ranger KMS out of /tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786289732823074-3431-0/minicluster-data/ranger-kms
I20260809 15:37:12.333199 3431 mini_ranger_kms.cc:227] Using postgres at 127.3.89.212:34095
2026-08-09 15:37:12,526 [I] DB FLAVOR :POSTGRES
2026-08-09 15:37:12,527 [I] --------- Verifying Ranger DB connection ---------
2026-08-09 15:37:12,527 [I] Checking connection
2026-08-09 15:37:12,527 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskm314YR/build/debug/bin/postgresql.jar:/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786289732823074-3431-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.3.89.212:34095/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -query "SELECT 1;"
2026-08-09 15:37:12,793 [I] connection success
2026-08-09 15:37:12,793 [I] --------- Verifying Ranger DB tables ---------
2026-08-09 15:37:12,793 [I] Verifying table ranger_masterkey in database rangerkms
2026-08-09 15:37:12,793 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskm314YR/build/debug/bin/postgresql.jar:/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786289732823074-3431-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.3.89.212:34095/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='rangerkms' and table_name = 'ranger_masterkey') as temp;"
2026-08-09 15:37:13,073 [I] Table ranger_masterkey does not exist in database rangerkms
2026-08-09 15:37:13,074 [I] --------- Importing Ranger Core DB Schema ---------
2026-08-09 15:37:13,074 [I] Importing db schema to database rangerkms from file: kms_core_db_postgres.sql
2026-08-09 15:37:13,074 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskm314YR/build/debug/bin/postgresql.jar:/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786289732823074-3431-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.3.89.212:34095/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -input /tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786289732823074-3431-0/minicluster-data/ranger-kms/db/postgres/kms_core_db_postgres.sql
2026-08-09 15:37:13,347 [I] kms_core_db_postgres.sql DB schema imported successfully
I20260809 15:37:13.830139 3431 mini_ranger_kms.cc:310] Created kms service
I20260809 15:37:14.068147 3431 mini_ranger_kms.cc:326] Created kudu user
I20260809 15:37:14.109669 3431 mini_ranger_kms.cc:343] Created rangerkms user
Aug 09 15:37:14 dist-test-slave-2w3w krb5kdc[15003](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786289831, etypes {rep=17 tkt=17 ses=17}, keyadmin@KRBTEST.COM for HTTP/127.3.89.212@KRBTEST.COM
I20260809 15:37:14.493309 3431 mini_ranger_kms.cc:384] Added ranger policy
I20260809 15:37:14.493460 3431 mini_ranger_kms.cc:238] Using RangerKMS classpath: /tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786289732823074-3431-0/minicluster-data/ranger-kms:/tmp/dist-test-taskm314YR/build/debug/bin/postgresql.jar:/tmp/dist-test-taskm314YR/thirdparty/src/ranger-2.1.0-kms/ews/webapp/WEB-INF/classes/lib/*:/tmp/dist-test-taskm314YR/thirdparty/src/ranger-2.1.0-kms/ews/webapp/lib/*:/usr/lib/jvm/java-1.8.0-openjdk-amd64/lib/*
I20260809 15:37:14.493481 3431 mini_ranger_kms.cc:240] Using host: 127.3.89.212
I20260809 15:37:14.498018 3431 mini_ranger_kms.cc:276] Ranger KMS PID: 15422
I20260809 15:37:14.498157 3431 mini_ranger_kms.cc:277] Ranger KMS URL: http://127.3.89.212:54653
Aug 09, 2026 3:37:15 PM org.apache.ranger.server.tomcat.EmbeddedServer getKeyManagers
WARNING: Config 'ranger.keystore.file' or 'ranger.service.https.attrib.keystore.file' is not found or contains blank value
Aug 09, 2026 3:37:15 PM org.apache.ranger.server.tomcat.EmbeddedServer getTrustManagers
WARNING: Config 'ranger.truststore.file' is not found or contains blank value!
Aug 09, 2026 3:37:15 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Webapp file =/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786289732823074-3431-0/minicluster-data/ranger-kms/ews/webapp, webAppName = /kms
Aug 09, 2026 3:37:16 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Adding webapp [/kms] = path [/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786289732823074-3431-0/minicluster-data/ranger-kms/ews/webapp] .....
Aug 09, 2026 3:37:16 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Finished init of webapp [/kms] = path [/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786289732823074-3431-0/minicluster-data/ranger-kms/ews/webapp].
Aug 09, 2026 3:37:16 PM org.apache.ranger.server.tomcat.EmbeddedServer startServer
INFO: Server Name : minirangerkms
I20260809 15:37:16.912250 3431 mini_ranger_kms.cc:207] Time spent starting Ranger KMS: real 5.323s user 0.000s sys 0.181s
I20260809 15:37:16.912427 3431 mini_ranger_kms.cc:397] {"name":"kuduclusterkey","cipher":"AES/CTR/NoPadding","length":128,"description":"kuduclusterkey"}
I20260809 15:37:16.912566 3431 mini_ranger_kms.cc:401] 127.3.89.212:54653/kms/v1/keys
Aug 09 15:37:17 dist-test-slave-2w3w krb5kdc[15003](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1786289837, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.3.89.212@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Aug 09 15:37:17 dist-test-slave-2w3w krb5kdc[15003](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786289837, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.3.89.212@KRBTEST.COM for HTTP/127.3.89.212@KRBTEST.COM
Aug 09 15:37:17 dist-test-slave-2w3w krb5kdc[15003](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786289837, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.3.89.212@KRBTEST.COM for HTTP/127.3.89.212@KRBTEST.COM
Aug 09 15:37:20 dist-test-slave-2w3w krb5kdc[15003](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786289840, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
WARNING: no policy specified for kudu/127.3.89.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.3.89.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.3.89.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786289732823074-3431-0/krb5kdc/kudu_127.3.89.254.keytab.
Entry for principal kudu/127.3.89.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786289732823074-3431-0/krb5kdc/kudu_127.3.89.254.keytab.
I20260809 15:37:20.308954 3431 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskm314YR/build/debug/bin/kudu
/tmp/dist-test-taskm314YR/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786289732823074-3431-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786289732823074-3431-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786289732823074-3431-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786289732823074-3431-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.3.89.254:43043
--webserver_interface=127.3.89.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786289732823074-3431-0/krb5kdc/kudu_127.3.89.254.keytab
--principal=kudu/127.3.89.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.3.89.212:38959
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.3.89.254:43043
--ranger_config_path=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786289732823074-3431-0/minicluster-data/ranger-client
--trusted_user_acl=test-admin
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.3.89.212:54653/kms
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786289732823074-3431-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786289732823074-3431-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786289732823074-3431-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 15:37:20.419341 15477 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 15:37:20.419610 15477 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 15:37:20.419651 15477 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 15:37:20.423767 15477 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 15:37:20.423830 15477 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 15:37:20.423851 15477 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 15:37:20.423866 15477 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 15:37:20.423882 15477 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 15:37:20.429296 15477 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.3.89.212:38959
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786289732823074-3431-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786289732823074-3431-0/minicluster-data/master-0/wal
--ranger_kms_url=127.3.89.212:54653/kms
--trusted_user_acl=<redacted>
--ipki_ca_key_size=768
--master_addresses=127.3.89.254:43043
--ranger_config_path=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786289732823074-3431-0/minicluster-data/ranger-client
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786289732823074-3431-0/krb5kdc/kudu_127.3.89.254.keytab
--principal=kudu/127.3.89.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.3.89.254:43043
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786289732823074-3431-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.3.89.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.15477
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786289732823074-3431-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 09 Aug 2026 14:46:55 UTC on ccycloud.quasar-yswkmf.root.comops.site
build id 4576540
I20260809 15:37:20.430320 15477 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 15:37:20.431397 15477 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 15:37:20.440358 15482 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 15:37:20.440341 15485 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 15:37:20.440415 15477 server_base.cc:1034] running on GCE node
W20260809 15:37:20.440341 15483 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 15:37:20.441192 15477 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 15:37:20.442260 15477 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 15:37:20.443421 15477 hybrid_clock.cc:648] HybridClock initialized: now 1786289840443396 us; error 38 us; skew 500 ppm
Aug 09 15:37:20 dist-test-slave-2w3w krb5kdc[15003](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786289840, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.89.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 15:37:20.449011 15477 init.cc:377] Logged in from keytab as kudu/127.3.89.254@KRBTEST.COM (short username kudu)
I20260809 15:37:20.449987 15477 webserver.cc:460] Webserver started at http://127.3.89.254:35065/ using document root <none> and password file <none>
I20260809 15:37:20.450562 15477 fs_manager.cc:359] Metadata directory not provided
I20260809 15:37:20.450619 15477 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 15:37:20.450872 15477 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 09 15:37:20 dist-test-slave-2w3w krb5kdc[15003](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786289840, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.89.254@KRBTEST.COM for HTTP/127.3.89.212@KRBTEST.COM
I20260809 15:37:20.500005 15477 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786289732823074-3431-0/minicluster-data/master-0/data/instance:
uuid: "cfa3b33d5ca74124b71aac50c9fe2751"
format_stamp: "Formatted at 2026-08-09 15:37:20 on dist-test-slave-2w3w"
server_key: "aa4d2c8dad46d269ce58921c63ca50f5"
server_key_iv: "3f9ca964129ac542a26a08403a624dc9"
server_key_version: "kuduclusterkey@0"
I20260809 15:37:20.500775 15477 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786289732823074-3431-0/minicluster-data/master-0/wal/instance:
uuid: "cfa3b33d5ca74124b71aac50c9fe2751"
format_stamp: "Formatted at 2026-08-09 15:37:20 on dist-test-slave-2w3w"
server_key: "aa4d2c8dad46d269ce58921c63ca50f5"
server_key_iv: "3f9ca964129ac542a26a08403a624dc9"
server_key_version: "kuduclusterkey@0"
I20260809 15:37:20.504827 15477 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260809 15:37:20.523909 15492 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 15:37:20.525591 15477 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260809 15:37:20.525740 15477 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786289732823074-3431-0/minicluster-data/master-0/data,/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786289732823074-3431-0/minicluster-data/master-0/wal
uuid: "cfa3b33d5ca74124b71aac50c9fe2751"
format_stamp: "Formatted at 2026-08-09 15:37:20 on dist-test-slave-2w3w"
server_key: "aa4d2c8dad46d269ce58921c63ca50f5"
server_key_iv: "3f9ca964129ac542a26a08403a624dc9"
server_key_version: "kuduclusterkey@0"
I20260809 15:37:20.525895 15477 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786289732823074-3431-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786289732823074-3431-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786289732823074-3431-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 15:37:20.546911 15477 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 15:37:20.550145 15477 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 15:37:20.550348 15477 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 15:37:20.558702 15477 rpc_server.cc:307] RPC server started. Bound to: 127.3.89.254:43043
I20260809 15:37:20.558744 15549 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.3.89.254:43043 every 8 connection(s)
I20260809 15:37:20.559868 15477 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786289732823074-3431-0/minicluster-data/master-0/data/info.pb
I20260809 15:37:20.562646 15550 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 15:37:20.567175 3431 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskm314YR/build/debug/bin/kudu as pid 15477
I20260809 15:37:20.567304 3431 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786289732823074-3431-0/minicluster-data/master-0/wal/instance
I20260809 15:37:20.567597 3431 external_mini_cluster.cc:1428] Setting key 806706a7876cf843e472b83649e07adf
I20260809 15:37:20.568336 15550 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P cfa3b33d5ca74124b71aac50c9fe2751: Bootstrap starting.
I20260809 15:37:20.571099 15550 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P cfa3b33d5ca74124b71aac50c9fe2751: Neither blocks nor log segments found. Creating new log.
I20260809 15:37:20.572060 15550 log.cc:824] T 00000000000000000000000000000000 P cfa3b33d5ca74124b71aac50c9fe2751: Log is configured to *not* fsync() on all Append() calls
I20260809 15:37:20.574213 15550 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P cfa3b33d5ca74124b71aac50c9fe2751: No bootstrap required, opened a new log
Aug 09 15:37:20 dist-test-slave-2w3w krb5kdc[15003](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786289840, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.3.89.254@KRBTEST.COM
I20260809 15:37:20.578440 15550 raft_consensus.cc:348] T 00000000000000000000000000000000 P cfa3b33d5ca74124b71aac50c9fe2751 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cfa3b33d5ca74124b71aac50c9fe2751" member_type: VOTER last_known_addr { host: "127.3.89.254" port: 43043 } }
I20260809 15:37:20.578734 15550 raft_consensus.cc:374] T 00000000000000000000000000000000 P cfa3b33d5ca74124b71aac50c9fe2751 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 15:37:20.578860 15550 raft_consensus.cc:729] T 00000000000000000000000000000000 P cfa3b33d5ca74124b71aac50c9fe2751 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: cfa3b33d5ca74124b71aac50c9fe2751, State: Initialized, Role: FOLLOWER
I20260809 15:37:20.579429 15550 consensus_queue.cc:260] T 00000000000000000000000000000000 P cfa3b33d5ca74124b71aac50c9fe2751 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cfa3b33d5ca74124b71aac50c9fe2751" member_type: VOTER last_known_addr { host: "127.3.89.254" port: 43043 } }
I20260809 15:37:20.579617 15550 raft_consensus.cc:388] T 00000000000000000000000000000000 P cfa3b33d5ca74124b71aac50c9fe2751 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 15:37:20.579728 15550 raft_consensus.cc:482] T 00000000000000000000000000000000 P cfa3b33d5ca74124b71aac50c9fe2751 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 15:37:20.579849 15550 raft_consensus.cc:3037] T 00000000000000000000000000000000 P cfa3b33d5ca74124b71aac50c9fe2751 [term 0 FOLLOWER]: Advancing to term 1
I20260809 15:37:20.581149 15550 raft_consensus.cc:504] T 00000000000000000000000000000000 P cfa3b33d5ca74124b71aac50c9fe2751 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cfa3b33d5ca74124b71aac50c9fe2751" member_type: VOTER last_known_addr { host: "127.3.89.254" port: 43043 } }
I20260809 15:37:20.581542 15550 leader_election.cc:302] T 00000000000000000000000000000000 P cfa3b33d5ca74124b71aac50c9fe2751 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: cfa3b33d5ca74124b71aac50c9fe2751; no voters:
I20260809 15:37:20.581962 15550 leader_election.cc:288] T 00000000000000000000000000000000 P cfa3b33d5ca74124b71aac50c9fe2751 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 15:37:20.582072 15555 raft_consensus.cc:2781] T 00000000000000000000000000000000 P cfa3b33d5ca74124b71aac50c9fe2751 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 15:37:20.582351 15555 raft_consensus.cc:686] T 00000000000000000000000000000000 P cfa3b33d5ca74124b71aac50c9fe2751 [term 1 LEADER]: Becoming Leader. State: Replica: cfa3b33d5ca74124b71aac50c9fe2751, State: Running, Role: LEADER
I20260809 15:37:20.583348 15555 consensus_queue.cc:237] T 00000000000000000000000000000000 P cfa3b33d5ca74124b71aac50c9fe2751 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cfa3b33d5ca74124b71aac50c9fe2751" member_type: VOTER last_known_addr { host: "127.3.89.254" port: 43043 } }
I20260809 15:37:20.583437 15550 sys_catalog.cc:564] T 00000000000000000000000000000000 P cfa3b33d5ca74124b71aac50c9fe2751 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 15:37:20.584252 15553 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:37:20.569089 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45690 (local address 127.3.89.254:43043)
0809 15:37:20.569567 (+ 478us) server_negotiation.cc:206] Beginning negotiation
0809 15:37:20.569589 (+ 22us) server_negotiation.cc:399] Waiting for connection header
0809 15:37:20.569631 (+ 42us) server_negotiation.cc:407] Connection header received
0809 15:37:20.570425 (+ 794us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 15:37:20.570446 (+ 21us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 15:37:20.570839 (+ 393us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 15:37:20.571235 (+ 396us) server_negotiation.cc:226] Negotiated authn=SASL
0809 15:37:20.572111 (+ 876us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:37:20.573452 (+ 1341us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:37:20.574630 (+ 1178us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:37:20.575332 (+ 702us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:37:20.575492 (+ 160us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:37:20.578650 (+ 3158us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 15:37:20.578657 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 15:37:20.578672 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 15:37:20.578705 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0809 15:37:20.580808 (+ 2103us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 15:37:20.581329 (+ 521us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 15:37:20.581335 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 15:37:20.581341 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0809 15:37:20.581473 (+ 132us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 15:37:20.581786 (+ 313us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 15:37:20.581789 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 15:37:20.581791 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 15:37:20.582351 (+ 560us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 15:37:20.582520 (+ 169us) server_negotiation.cc:1035] Waiting for connection context
0809 15:37:20.582832 (+ 312us) server_negotiation.cc:299] Negotiation successful
0809 15:37:20.583320 (+ 488us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":314,"thread_start_us":130,"threads_started":1}
I20260809 15:37:20.585789 15550 ranger_client.cc:318] Using new properties file: /tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786289732823074-3431-0/minicluster-data/master-0/logs/kudu-ranger-subprocess-log4j2.properties
I20260809 15:37:20.585829 15556 sys_catalog.cc:455] T 00000000000000000000000000000000 P cfa3b33d5ca74124b71aac50c9fe2751 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "cfa3b33d5ca74124b71aac50c9fe2751" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cfa3b33d5ca74124b71aac50c9fe2751" member_type: VOTER last_known_addr { host: "127.3.89.254" port: 43043 } } }
I20260809 15:37:20.585950 15556 sys_catalog.cc:458] T 00000000000000000000000000000000 P cfa3b33d5ca74124b71aac50c9fe2751 [sys.catalog]: This master's current role is: LEADER
I20260809 15:37:20.586254 15557 sys_catalog.cc:455] T 00000000000000000000000000000000 P cfa3b33d5ca74124b71aac50c9fe2751 [sys.catalog]: SysCatalogTable state changed. Reason: New leader cfa3b33d5ca74124b71aac50c9fe2751. Latest consensus state: current_term: 1 leader_uuid: "cfa3b33d5ca74124b71aac50c9fe2751" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cfa3b33d5ca74124b71aac50c9fe2751" member_type: VOTER last_known_addr { host: "127.3.89.254" port: 43043 } } }
I20260809 15:37:20.586371 15557 sys_catalog.cc:458] T 00000000000000000000000000000000 P cfa3b33d5ca74124b71aac50c9fe2751 [sys.catalog]: This master's current role is: LEADER
I20260809 15:37:20.588081 15569 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 15:37:20.593951 15569 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 15:37:20.603441 15569 catalog_manager.cc:1347] Generated new cluster ID: 555c52e0e5884872a2bd69c41061b37e
I20260809 15:37:20.603555 15569 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 15:37:20.647850 15569 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 15:37:20.649266 15569 catalog_manager.cc:1504] Loading token signing keys...
I20260809 15:37:20.653890 15569 catalog_manager.cc:5868] T 00000000000000000000000000000000 P cfa3b33d5ca74124b71aac50c9fe2751: Generated new TSK 0
I20260809 15:37:20.654769 15569 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.3.89.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.3.89.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.3.89.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786289732823074-3431-0/krb5kdc/kudu_127.3.89.193.keytab.
Entry for principal kudu/127.3.89.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786289732823074-3431-0/krb5kdc/kudu_127.3.89.193.keytab.
I20260809 15:37:21.337936 3431 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskm314YR/build/debug/bin/kudu
/tmp/dist-test-taskm314YR/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786289732823074-3431-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786289732823074-3431-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786289732823074-3431-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786289732823074-3431-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.3.89.212:54653/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.3.89.193:0
--local_ip_for_outbound_sockets=127.3.89.193
--webserver_interface=127.3.89.193
--webserver_port=0
--tserver_master_addrs=127.3.89.254:43043
--keytab_file=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786289732823074-3431-0/krb5kdc/kudu_127.3.89.193.keytab
--principal=kudu/127.3.89.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.3.89.212:38959
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786289732823074-3431-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786289732823074-3431-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786289732823074-3431-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 15:37:21.475402 15597 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 15:37:21.475666 15597 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 15:37:21.475765 15597 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 15:37:21.480099 15597 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 15:37:21.480177 15597 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 15:37:21.480278 15597 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.3.89.193
I20260809 15:37:21.485795 15597 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.3.89.212:38959
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786289732823074-3431-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786289732823074-3431-0/minicluster-data/ts-0/wal
--ranger_kms_url=127.3.89.212:54653/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786289732823074-3431-0/krb5kdc/kudu_127.3.89.193.keytab
--principal=kudu/127.3.89.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.3.89.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786289732823074-3431-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.3.89.193
--webserver_port=0
--tserver_master_addrs=127.3.89.254:43043
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.15597
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.3.89.193
--log_dir=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786289732823074-3431-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 09 Aug 2026 14:46:55 UTC on ccycloud.quasar-yswkmf.root.comops.site
build id 4576540
I20260809 15:37:21.486855 15597 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 15:37:21.487910 15597 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 15:37:21.498885 15605 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 15:37:21.498888 15604 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 15:37:21.499089 15607 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 15:37:21.500275 15597 server_base.cc:1034] running on GCE node
I20260809 15:37:21.500891 15597 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 15:37:21.501771 15597 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 15:37:21.503010 15597 hybrid_clock.cc:648] HybridClock initialized: now 1786289841502972 us; error 49 us; skew 500 ppm
Aug 09 15:37:21 dist-test-slave-2w3w krb5kdc[15003](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786289841, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.89.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 15:37:21.506897 15597 init.cc:377] Logged in from keytab as kudu/127.3.89.193@KRBTEST.COM (short username kudu)
I20260809 15:37:21.508028 15597 webserver.cc:460] Webserver started at http://127.3.89.193:37669/ using document root <none> and password file <none>
I20260809 15:37:21.508770 15597 fs_manager.cc:359] Metadata directory not provided
I20260809 15:37:21.508870 15597 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 15:37:21.509104 15597 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 09 15:37:21 dist-test-slave-2w3w krb5kdc[15003](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786289841, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.89.193@KRBTEST.COM for HTTP/127.3.89.212@KRBTEST.COM
I20260809 15:37:21.530045 15597 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786289732823074-3431-0/minicluster-data/ts-0/data/instance:
uuid: "428bbceec5914c4b8c7f64c5fd848ac6"
format_stamp: "Formatted at 2026-08-09 15:37:21 on dist-test-slave-2w3w"
server_key: "a608139a8a9140489329782e026eca0b"
server_key_iv: "c520581272f194a6037c857b17ae4a54"
server_key_version: "kuduclusterkey@0"
I20260809 15:37:21.530891 15597 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786289732823074-3431-0/minicluster-data/ts-0/wal/instance:
uuid: "428bbceec5914c4b8c7f64c5fd848ac6"
format_stamp: "Formatted at 2026-08-09 15:37:21 on dist-test-slave-2w3w"
server_key: "a608139a8a9140489329782e026eca0b"
server_key_iv: "c520581272f194a6037c857b17ae4a54"
server_key_version: "kuduclusterkey@0"
I20260809 15:37:21.538929 15597 fs_manager.cc:691] Time spent creating directory manager: real 0.008s user 0.003s sys 0.001s
I20260809 15:37:21.571385 15614 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 15:37:21.573940 15597 fs_manager.cc:725] Time spent opening block manager: real 0.019s user 0.003s sys 0.000s
I20260809 15:37:21.574079 15597 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786289732823074-3431-0/minicluster-data/ts-0/data,/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786289732823074-3431-0/minicluster-data/ts-0/wal
uuid: "428bbceec5914c4b8c7f64c5fd848ac6"
format_stamp: "Formatted at 2026-08-09 15:37:21 on dist-test-slave-2w3w"
server_key: "a608139a8a9140489329782e026eca0b"
server_key_iv: "c520581272f194a6037c857b17ae4a54"
server_key_version: "kuduclusterkey@0"
I20260809 15:37:21.574220 15597 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786289732823074-3431-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786289732823074-3431-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786289732823074-3431-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 15:37:21.599072 15597 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 15:37:21.602768 15597 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 15:37:21.602999 15597 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 15:37:21.603714 15597 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 15:37:21.604768 15597 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 15:37:21.604851 15597 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 15:37:21.604912 15597 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 15:37:21.604943 15597 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 15:37:21.617432 15597 rpc_server.cc:307] RPC server started. Bound to: 127.3.89.193:43925
I20260809 15:37:21.617591 15733 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.3.89.193:43925 every 8 connection(s)
I20260809 15:37:21.618619 15597 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786289732823074-3431-0/minicluster-data/ts-0/data/info.pb
I20260809 15:37:21.627559 3431 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskm314YR/build/debug/bin/kudu as pid 15597
I20260809 15:37:21.627691 3431 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786289732823074-3431-0/minicluster-data/ts-0/wal/instance
I20260809 15:37:21.628007 3431 external_mini_cluster.cc:1428] Setting key 8c2239b0a0bb6a62b90352042844e021
Aug 09 15:37:21 dist-test-slave-2w3w krb5kdc[15003](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786289841, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.89.193@KRBTEST.COM for kudu/127.3.89.254@KRBTEST.COM
I20260809 15:37:21.638983 15739 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:37:21.621363 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.89.254:43043 (local address 127.3.89.193:47015)
0809 15:37:21.621953 (+ 590us) negotiation.cc:107] Waiting for socket to connect
0809 15:37:21.622014 (+ 61us) client_negotiation.cc:174] Beginning negotiation
0809 15:37:21.623243 (+ 1229us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 15:37:21.623911 (+ 668us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 15:37:21.623921 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 15:37:21.624306 (+ 385us) client_negotiation.cc:189] Negotiated authn=SASL
0809 15:37:21.625013 (+ 707us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:37:21.625038 (+ 25us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:37:21.626256 (+ 1218us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:37:21.626260 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:37:21.627909 (+ 1649us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:37:21.627921 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:37:21.629037 (+ 1116us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:37:21.629041 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:37:21.629249 (+ 208us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:37:21.631125 (+ 1876us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 15:37:21.631188 (+ 63us) client_negotiation.cc:656] calling sasl_client_start()
0809 15:37:21.633068 (+ 1880us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 15:37:21.635517 (+ 2449us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 15:37:21.635525 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 15:37:21.635541 (+ 16us) client_negotiation.cc:760] calling sasl_client_step()
0809 15:37:21.635892 (+ 351us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 15:37:21.636244 (+ 352us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 15:37:21.636247 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 15:37:21.636250 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0809 15:37:21.636440 (+ 190us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 15:37:21.636961 (+ 521us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 15:37:21.636969 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 15:37:21.637316 (+ 347us) client_negotiation.cc:769] Sending connection context
0809 15:37:21.637531 (+ 215us) client_negotiation.cc:240] Negotiation successful
0809 15:37:21.637883 (+ 352us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":345,"thread_start_us":156,"threads_started":1}
I20260809 15:37:21.639752 15736 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:37:21.620907 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.89.193:47015 (local address 127.3.89.254:43043)
0809 15:37:21.621217 (+ 310us) server_negotiation.cc:206] Beginning negotiation
0809 15:37:21.621226 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0809 15:37:21.622134 (+ 908us) server_negotiation.cc:407] Connection header received
0809 15:37:21.623526 (+ 1392us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 15:37:21.623530 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 15:37:21.623592 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 15:37:21.623707 (+ 115us) server_negotiation.cc:226] Negotiated authn=SASL
0809 15:37:21.625207 (+ 1500us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:37:21.626100 (+ 893us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:37:21.628070 (+ 1970us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:37:21.628486 (+ 416us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:37:21.628575 (+ 89us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:37:21.633274 (+ 4699us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 15:37:21.633278 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 15:37:21.633281 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 15:37:21.633309 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0809 15:37:21.635381 (+ 2072us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 15:37:21.636019 (+ 638us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 15:37:21.636023 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 15:37:21.636025 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 15:37:21.636122 (+ 97us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 15:37:21.636550 (+ 428us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 15:37:21.636554 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 15:37:21.636555 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 15:37:21.636788 (+ 233us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 15:37:21.636901 (+ 113us) server_negotiation.cc:1035] Waiting for connection context
0809 15:37:21.639448 (+ 2547us) server_negotiation.cc:299] Negotiation successful
0809 15:37:21.639579 (+ 131us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":229,"thread_start_us":115,"threads_started":1}
I20260809 15:37:21.641369 15734 heartbeater.cc:347] Connected to a master server at 127.3.89.254:43043
I20260809 15:37:21.641606 15734 heartbeater.cc:464] Registering TS with master...
WARNING: no policy specified for kudu/127.3.89.194@KRBTEST.COM; defaulting to no policy
I20260809 15:37:21.642202 15734 heartbeater.cc:511] Master 127.3.89.254:43043 requested a full tablet report, sending...
I20260809 15:37:21.643651 15509 ts_manager.cc:194] Registered new tserver with Master: 428bbceec5914c4b8c7f64c5fd848ac6 (127.3.89.193:43925)
I20260809 15:37:21.644749 15509 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.3.89.193@KRBTEST.COM'} at 127.3.89.193:47015
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.3.89.194@KRBTEST.COM" created.
Aug 09 15:37:21 dist-test-slave-2w3w krb5kdc[15003](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1786289841, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.89.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.3.89.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786289732823074-3431-0/krb5kdc/kudu_127.3.89.194.keytab.
Entry for principal kudu/127.3.89.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786289732823074-3431-0/krb5kdc/kudu_127.3.89.194.keytab.
I20260809 15:37:21.668483 3431 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskm314YR/build/debug/bin/kudu
/tmp/dist-test-taskm314YR/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786289732823074-3431-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786289732823074-3431-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786289732823074-3431-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786289732823074-3431-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.3.89.212:54653/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.3.89.194:0
--local_ip_for_outbound_sockets=127.3.89.194
--webserver_interface=127.3.89.194
--webserver_port=0
--tserver_master_addrs=127.3.89.254:43043
--keytab_file=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786289732823074-3431-0/krb5kdc/kudu_127.3.89.194.keytab
--principal=kudu/127.3.89.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.3.89.212:38959
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786289732823074-3431-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786289732823074-3431-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786289732823074-3431-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 15:37:21.798712 15742 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 15:37:21.799006 15742 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 15:37:21.799100 15742 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 15:37:21.803118 15742 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 15:37:21.803192 15742 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 15:37:21.803293 15742 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.3.89.194
I20260809 15:37:21.808928 15742 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.3.89.212:38959
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786289732823074-3431-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786289732823074-3431-0/minicluster-data/ts-1/wal
--ranger_kms_url=127.3.89.212:54653/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786289732823074-3431-0/krb5kdc/kudu_127.3.89.194.keytab
--principal=kudu/127.3.89.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.3.89.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786289732823074-3431-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.3.89.194
--webserver_port=0
--tserver_master_addrs=127.3.89.254:43043
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.15742
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.3.89.194
--log_dir=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786289732823074-3431-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 09 Aug 2026 14:46:55 UTC on ccycloud.quasar-yswkmf.root.comops.site
build id 4576540
I20260809 15:37:21.810210 15742 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 15:37:21.811280 15742 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 15:37:21.822265 15751 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 15:37:21.822412 15749 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 15:37:21.822275 15748 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 15:37:21.823793 15742 server_base.cc:1034] running on GCE node
I20260809 15:37:21.824291 15742 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 15:37:21.825062 15742 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 15:37:21.826244 15742 hybrid_clock.cc:648] HybridClock initialized: now 1786289841826216 us; error 49 us; skew 500 ppm
Aug 09 15:37:21 dist-test-slave-2w3w krb5kdc[15003](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786289841, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.89.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 15:37:21.829845 15742 init.cc:377] Logged in from keytab as kudu/127.3.89.194@KRBTEST.COM (short username kudu)
I20260809 15:37:21.830965 15742 webserver.cc:460] Webserver started at http://127.3.89.194:36241/ using document root <none> and password file <none>
I20260809 15:37:21.831609 15742 fs_manager.cc:359] Metadata directory not provided
I20260809 15:37:21.831700 15742 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 15:37:21.831935 15742 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 09 15:37:21 dist-test-slave-2w3w krb5kdc[15003](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786289841, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.89.194@KRBTEST.COM for HTTP/127.3.89.212@KRBTEST.COM
I20260809 15:37:21.855822 15742 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786289732823074-3431-0/minicluster-data/ts-1/data/instance:
uuid: "deeb678a8de14a2a80496d2867663ad6"
format_stamp: "Formatted at 2026-08-09 15:37:21 on dist-test-slave-2w3w"
server_key: "55c2ef51539bef15738183d8c12e0041"
server_key_iv: "7187a580fc6a50a6295f2835d1547737"
server_key_version: "kuduclusterkey@0"
I20260809 15:37:21.856634 15742 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786289732823074-3431-0/minicluster-data/ts-1/wal/instance:
uuid: "deeb678a8de14a2a80496d2867663ad6"
format_stamp: "Formatted at 2026-08-09 15:37:21 on dist-test-slave-2w3w"
server_key: "55c2ef51539bef15738183d8c12e0041"
server_key_iv: "7187a580fc6a50a6295f2835d1547737"
server_key_version: "kuduclusterkey@0"
I20260809 15:37:21.863626 15742 fs_manager.cc:691] Time spent creating directory manager: real 0.007s user 0.002s sys 0.004s
I20260809 15:37:21.881614 15758 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 15:37:21.883397 15742 fs_manager.cc:725] Time spent opening block manager: real 0.006s user 0.002s sys 0.000s
I20260809 15:37:21.883543 15742 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786289732823074-3431-0/minicluster-data/ts-1/data,/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786289732823074-3431-0/minicluster-data/ts-1/wal
uuid: "deeb678a8de14a2a80496d2867663ad6"
format_stamp: "Formatted at 2026-08-09 15:37:21 on dist-test-slave-2w3w"
server_key: "55c2ef51539bef15738183d8c12e0041"
server_key_iv: "7187a580fc6a50a6295f2835d1547737"
server_key_version: "kuduclusterkey@0"
I20260809 15:37:21.883692 15742 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786289732823074-3431-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786289732823074-3431-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786289732823074-3431-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 15:37:21.903239 15742 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 15:37:21.906809 15742 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 15:37:21.907034 15742 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 15:37:21.907739 15742 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 15:37:21.908885 15742 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 15:37:21.908962 15742 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 15:37:21.909024 15742 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 15:37:21.909055 15742 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 15:37:21.920024 15742 rpc_server.cc:307] RPC server started. Bound to: 127.3.89.194:39039
I20260809 15:37:21.921212 15742 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786289732823074-3431-0/minicluster-data/ts-1/data/info.pb
I20260809 15:37:21.926088 15878 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.3.89.194:39039 every 8 connection(s)
I20260809 15:37:21.930100 3431 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskm314YR/build/debug/bin/kudu as pid 15742
I20260809 15:37:21.930243 3431 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786289732823074-3431-0/minicluster-data/ts-1/wal/instance
I20260809 15:37:21.930627 3431 external_mini_cluster.cc:1428] Setting key 7fe8c57b79b1c53f59aba9f2eb042a6b
Aug 09 15:37:21 dist-test-slave-2w3w krb5kdc[15003](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786289841, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.89.194@KRBTEST.COM for kudu/127.3.89.254@KRBTEST.COM
I20260809 15:37:21.948366 15736 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:37:21.931171 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.89.194:38121 (local address 127.3.89.254:43043)
0809 15:37:21.931330 (+ 159us) server_negotiation.cc:206] Beginning negotiation
0809 15:37:21.931343 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0809 15:37:21.931824 (+ 481us) server_negotiation.cc:407] Connection header received
0809 15:37:21.935738 (+ 3914us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 15:37:21.935741 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 15:37:21.935795 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 15:37:21.935846 (+ 51us) server_negotiation.cc:226] Negotiated authn=SASL
0809 15:37:21.937617 (+ 1771us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:37:21.938456 (+ 839us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:37:21.940234 (+ 1778us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:37:21.940596 (+ 362us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:37:21.940681 (+ 85us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:37:21.943953 (+ 3272us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 15:37:21.943957 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 15:37:21.943960 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 15:37:21.943988 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0809 15:37:21.945830 (+ 1842us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 15:37:21.946470 (+ 640us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 15:37:21.946474 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 15:37:21.946475 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 15:37:21.946567 (+ 92us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 15:37:21.947005 (+ 438us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 15:37:21.947008 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 15:37:21.947010 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 15:37:21.947231 (+ 221us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 15:37:21.947317 (+ 86us) server_negotiation.cc:1035] Waiting for connection context
0809 15:37:21.948028 (+ 711us) server_negotiation.cc:299] Negotiation successful
0809 15:37:21.948180 (+ 152us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":78}
I20260809 15:37:21.950062 15881 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:37:21.928724 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.89.254:43043 (local address 127.3.89.194:38121)
0809 15:37:21.931657 (+ 2933us) negotiation.cc:107] Waiting for socket to connect
0809 15:37:21.931720 (+ 63us) client_negotiation.cc:174] Beginning negotiation
0809 15:37:21.935466 (+ 3746us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 15:37:21.936286 (+ 820us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 15:37:21.936295 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 15:37:21.936740 (+ 445us) client_negotiation.cc:189] Negotiated authn=SASL
0809 15:37:21.937431 (+ 691us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:37:21.937453 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:37:21.938622 (+ 1169us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:37:21.938625 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:37:21.940088 (+ 1463us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:37:21.940099 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:37:21.940890 (+ 791us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:37:21.940894 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:37:21.941089 (+ 195us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:37:21.941825 (+ 736us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 15:37:21.941848 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0809 15:37:21.943717 (+ 1869us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 15:37:21.945974 (+ 2257us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 15:37:21.945979 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 15:37:21.945995 (+ 16us) client_negotiation.cc:760] calling sasl_client_step()
0809 15:37:21.946323 (+ 328us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 15:37:21.946675 (+ 352us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 15:37:21.946678 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 15:37:21.946680 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 15:37:21.946861 (+ 181us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 15:37:21.947399 (+ 538us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 15:37:21.947405 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 15:37:21.947710 (+ 305us) client_negotiation.cc:769] Sending connection context
0809 15:37:21.948620 (+ 910us) client_negotiation.cc:240] Negotiation successful
0809 15:37:21.948950 (+ 330us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":2711,"thread_start_us":163,"threads_started":1}
I20260809 15:37:21.951357 15879 heartbeater.cc:347] Connected to a master server at 127.3.89.254:43043
I20260809 15:37:21.951610 15879 heartbeater.cc:464] Registering TS with master...
I20260809 15:37:21.952174 15879 heartbeater.cc:511] Master 127.3.89.254:43043 requested a full tablet report, sending...
I20260809 15:37:21.953281 15509 ts_manager.cc:194] Registered new tserver with Master: deeb678a8de14a2a80496d2867663ad6 (127.3.89.194:39039)
I20260809 15:37:21.954063 15509 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.3.89.194@KRBTEST.COM'} at 127.3.89.194:38121
WARNING: no policy specified for kudu/127.3.89.195@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.3.89.195@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.3.89.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786289732823074-3431-0/krb5kdc/kudu_127.3.89.195.keytab.
Entry for principal kudu/127.3.89.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786289732823074-3431-0/krb5kdc/kudu_127.3.89.195.keytab.
I20260809 15:37:21.975993 3431 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskm314YR/build/debug/bin/kudu
/tmp/dist-test-taskm314YR/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786289732823074-3431-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786289732823074-3431-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786289732823074-3431-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786289732823074-3431-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.3.89.212:54653/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.3.89.195:0
--local_ip_for_outbound_sockets=127.3.89.195
--webserver_interface=127.3.89.195
--webserver_port=0
--tserver_master_addrs=127.3.89.254:43043
--keytab_file=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786289732823074-3431-0/krb5kdc/kudu_127.3.89.195.keytab
--principal=kudu/127.3.89.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.3.89.212:38959
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786289732823074-3431-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786289732823074-3431-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786289732823074-3431-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 15:37:22.137296 15884 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 15:37:22.138088 15884 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 15:37:22.138234 15884 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 15:37:22.144400 15884 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 15:37:22.144583 15884 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 15:37:22.144738 15884 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.3.89.195
I20260809 15:37:22.152580 15884 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.3.89.212:38959
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786289732823074-3431-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786289732823074-3431-0/minicluster-data/ts-2/wal
--ranger_kms_url=127.3.89.212:54653/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786289732823074-3431-0/krb5kdc/kudu_127.3.89.195.keytab
--principal=kudu/127.3.89.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.3.89.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786289732823074-3431-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.3.89.195
--webserver_port=0
--tserver_master_addrs=127.3.89.254:43043
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.15884
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.3.89.195
--log_dir=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786289732823074-3431-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 09 Aug 2026 14:46:55 UTC on ccycloud.quasar-yswkmf.root.comops.site
build id 4576540
I20260809 15:37:22.154129 15884 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 15:37:22.155555 15884 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260809 15:37:22.168915 15884 server_base.cc:1034] running on GCE node
W20260809 15:37:22.169366 15889 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 15:37:22.170042 15893 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 15:37:22.170331 15890 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 15:37:22.170866 15884 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 15:37:22.171753 15884 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 15:37:22.172955 15884 hybrid_clock.cc:648] HybridClock initialized: now 1786289842172881 us; error 89 us; skew 500 ppm
Aug 09 15:37:22 dist-test-slave-2w3w krb5kdc[15003](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786289842, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.89.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 15:37:22.176640 15884 init.cc:377] Logged in from keytab as kudu/127.3.89.195@KRBTEST.COM (short username kudu)
I20260809 15:37:22.177704 15884 webserver.cc:460] Webserver started at http://127.3.89.195:35003/ using document root <none> and password file <none>
I20260809 15:37:22.178416 15884 fs_manager.cc:359] Metadata directory not provided
I20260809 15:37:22.178503 15884 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 15:37:22.178746 15884 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 09 15:37:22 dist-test-slave-2w3w krb5kdc[15003](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786289842, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.89.195@KRBTEST.COM for HTTP/127.3.89.212@KRBTEST.COM
I20260809 15:37:22.193673 15884 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786289732823074-3431-0/minicluster-data/ts-2/data/instance:
uuid: "fc960bf7e36c4f1a807a66332168b814"
format_stamp: "Formatted at 2026-08-09 15:37:22 on dist-test-slave-2w3w"
server_key: "fba45f4258495919cc45c2f033f9192e"
server_key_iv: "27693be06a8dd484824b646ec18223f6"
server_key_version: "kuduclusterkey@0"
I20260809 15:37:22.194393 15884 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786289732823074-3431-0/minicluster-data/ts-2/wal/instance:
uuid: "fc960bf7e36c4f1a807a66332168b814"
format_stamp: "Formatted at 2026-08-09 15:37:22 on dist-test-slave-2w3w"
server_key: "fba45f4258495919cc45c2f033f9192e"
server_key_iv: "27693be06a8dd484824b646ec18223f6"
server_key_version: "kuduclusterkey@0"
I20260809 15:37:22.208652 15884 fs_manager.cc:691] Time spent creating directory manager: real 0.014s user 0.003s sys 0.004s
I20260809 15:37:22.229902 15900 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 15:37:22.231614 15884 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.002s sys 0.000s
I20260809 15:37:22.231750 15884 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786289732823074-3431-0/minicluster-data/ts-2/data,/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786289732823074-3431-0/minicluster-data/ts-2/wal
uuid: "fc960bf7e36c4f1a807a66332168b814"
format_stamp: "Formatted at 2026-08-09 15:37:22 on dist-test-slave-2w3w"
server_key: "fba45f4258495919cc45c2f033f9192e"
server_key_iv: "27693be06a8dd484824b646ec18223f6"
server_key_version: "kuduclusterkey@0"
I20260809 15:37:22.231878 15884 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786289732823074-3431-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786289732823074-3431-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786289732823074-3431-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
Aug 09 15:37:22 dist-test-slave-2w3w krb5kdc[15003](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786289841, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.89.254@KRBTEST.COM for HTTP/127.3.89.212@KRBTEST.COM
I20260809 15:37:22.259163 15884 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 15:37:22.262743 15884 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 15:37:22.263064 15884 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 15:37:22.263799 15884 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 15:37:22.264887 15884 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 15:37:22.265012 15884 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 15:37:22.265128 15884 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 15:37:22.265198 15884 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 15:37:22.286078 15884 rpc_server.cc:307] RPC server started. Bound to: 127.3.89.195:40181
I20260809 15:37:22.287379 15884 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786289732823074-3431-0/minicluster-data/ts-2/data/info.pb
I20260809 15:37:22.288095 3431 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskm314YR/build/debug/bin/kudu as pid 15884
I20260809 15:37:22.288228 3431 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786289732823074-3431-0/minicluster-data/ts-2/wal/instance
I20260809 15:37:22.288533 3431 external_mini_cluster.cc:1428] Setting key d18e756872637333e66fe8da19d33304
I20260809 15:37:22.291764 16020 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.3.89.195:40181 every 8 connection(s)
Aug 09 15:37:22 dist-test-slave-2w3w krb5kdc[15003](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786289842, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.89.195@KRBTEST.COM for kudu/127.3.89.254@KRBTEST.COM
I20260809 15:37:22.315021 15736 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:37:22.295416 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.89.195:47807 (local address 127.3.89.254:43043)
0809 15:37:22.295549 (+ 133us) server_negotiation.cc:206] Beginning negotiation
0809 15:37:22.295554 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 15:37:22.296222 (+ 668us) server_negotiation.cc:407] Connection header received
0809 15:37:22.297372 (+ 1150us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 15:37:22.297376 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 15:37:22.297426 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 15:37:22.297520 (+ 94us) server_negotiation.cc:226] Negotiated authn=SASL
0809 15:37:22.304724 (+ 7204us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:37:22.305598 (+ 874us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:37:22.307555 (+ 1957us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:37:22.307879 (+ 324us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:37:22.307953 (+ 74us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:37:22.310918 (+ 2965us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 15:37:22.310922 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 15:37:22.310925 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 15:37:22.310953 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0809 15:37:22.312672 (+ 1719us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 15:37:22.313258 (+ 586us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 15:37:22.313262 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 15:37:22.313264 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 15:37:22.313358 (+ 94us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 15:37:22.313733 (+ 375us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 15:37:22.313736 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 15:37:22.313737 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 15:37:22.313946 (+ 209us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 15:37:22.314022 (+ 76us) server_negotiation.cc:1035] Waiting for connection context
0809 15:37:22.314678 (+ 656us) server_negotiation.cc:299] Negotiation successful
0809 15:37:22.314829 (+ 151us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":36}
I20260809 15:37:22.316813 16024 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:37:22.294362 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.89.254:43043 (local address 127.3.89.195:47807)
0809 15:37:22.296080 (+ 1718us) negotiation.cc:107] Waiting for socket to connect
0809 15:37:22.296136 (+ 56us) client_negotiation.cc:174] Beginning negotiation
0809 15:37:22.297152 (+ 1016us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 15:37:22.303246 (+ 6094us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 15:37:22.303257 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 15:37:22.303663 (+ 406us) client_negotiation.cc:189] Negotiated authn=SASL
0809 15:37:22.304525 (+ 862us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:37:22.304552 (+ 27us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:37:22.305986 (+ 1434us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:37:22.305989 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:37:22.307417 (+ 1428us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:37:22.307428 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:37:22.308152 (+ 724us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:37:22.308156 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:37:22.308362 (+ 206us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:37:22.309088 (+ 726us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 15:37:22.309108 (+ 20us) client_negotiation.cc:656] calling sasl_client_start()
0809 15:37:22.310732 (+ 1624us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 15:37:22.312800 (+ 2068us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 15:37:22.312807 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 15:37:22.312824 (+ 17us) client_negotiation.cc:760] calling sasl_client_step()
0809 15:37:22.313147 (+ 323us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 15:37:22.313457 (+ 310us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 15:37:22.313460 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 15:37:22.313462 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 15:37:22.313630 (+ 168us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 15:37:22.314084 (+ 454us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 15:37:22.314090 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 15:37:22.314401 (+ 311us) client_negotiation.cc:769] Sending connection context
0809 15:37:22.315456 (+ 1055us) client_negotiation.cc:240] Negotiation successful
0809 15:37:22.315792 (+ 336us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1489,"thread_start_us":131,"threads_started":1}
I20260809 15:37:22.318219 16021 heartbeater.cc:347] Connected to a master server at 127.3.89.254:43043
I20260809 15:37:22.318467 16021 heartbeater.cc:464] Registering TS with master...
I20260809 15:37:22.319095 16021 heartbeater.cc:511] Master 127.3.89.254:43043 requested a full tablet report, sending...
I20260809 15:37:22.320130 15509 ts_manager.cc:194] Registered new tserver with Master: fc960bf7e36c4f1a807a66332168b814 (127.3.89.195:40181)
I20260809 15:37:22.320824 15509 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.3.89.195@KRBTEST.COM'} at 127.3.89.195:47807
I20260809 15:37:22.325652 3431 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260809 15:37:22.351719 15736 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:37:22.335983 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45704 (local address 127.3.89.254:43043)
0809 15:37:22.336125 (+ 142us) server_negotiation.cc:206] Beginning negotiation
0809 15:37:22.336129 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 15:37:22.342092 (+ 5963us) server_negotiation.cc:407] Connection header received
0809 15:37:22.342305 (+ 213us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 15:37:22.342309 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 15:37:22.342374 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 15:37:22.342428 (+ 54us) server_negotiation.cc:226] Negotiated authn=SASL
0809 15:37:22.343513 (+ 1085us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:37:22.344296 (+ 783us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:37:22.345603 (+ 1307us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:37:22.345914 (+ 311us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:37:22.345990 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:37:22.347958 (+ 1968us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 15:37:22.347962 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 15:37:22.347965 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 15:37:22.347995 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0809 15:37:22.349768 (+ 1773us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 15:37:22.350273 (+ 505us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 15:37:22.350277 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 15:37:22.350279 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 15:37:22.350376 (+ 97us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 15:37:22.350686 (+ 310us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 15:37:22.350689 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 15:37:22.350691 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 15:37:22.350907 (+ 216us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 15:37:22.351049 (+ 142us) server_negotiation.cc:1035] Waiting for connection context
0809 15:37:22.351419 (+ 370us) server_negotiation.cc:299] Negotiation successful
0809 15:37:22.351547 (+ 128us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
I20260809 15:37:22.355759 15509 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:45704:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260809 15:37:22.360467 15509 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260809 15:37:22.384690 16037 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:37:22.374063 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.89.194:39039 (local address 127.0.0.1:51042)
0809 15:37:22.376922 (+ 2859us) negotiation.cc:107] Waiting for socket to connect
0809 15:37:22.376936 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0809 15:37:22.377014 (+ 78us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 15:37:22.377924 (+ 910us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 15:37:22.377930 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 15:37:22.377951 (+ 21us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 15:37:22.378074 (+ 123us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:37:22.378079 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:37:22.380606 (+ 2527us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:37:22.380610 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:37:22.382725 (+ 2115us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:37:22.382736 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:37:22.384139 (+ 1403us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:37:22.384145 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:37:22.384261 (+ 116us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:37:22.384301 (+ 40us) client_negotiation.cc:769] Sending connection context
0809 15:37:22.384409 (+ 108us) client_negotiation.cc:240] Negotiation successful
0809 15:37:22.384533 (+ 124us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":2800,"mutex_wait_us":1,"thread_start_us":85,"threads_started":1}
I20260809 15:37:22.386402 16038 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:37:22.374470 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51042 (local address 127.3.89.194:39039)
0809 15:37:22.377264 (+ 2794us) server_negotiation.cc:206] Beginning negotiation
0809 15:37:22.377276 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0809 15:37:22.377301 (+ 25us) server_negotiation.cc:407] Connection header received
0809 15:37:22.377388 (+ 87us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 15:37:22.377393 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 15:37:22.377605 (+ 212us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 15:37:22.377754 (+ 149us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 15:37:22.379163 (+ 1409us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:37:22.380475 (+ 1312us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:37:22.383028 (+ 2553us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:37:22.383918 (+ 890us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:37:22.384013 (+ 95us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:37:22.386063 (+ 2050us) server_negotiation.cc:1035] Waiting for connection context
0809 15:37:22.386123 (+ 60us) server_negotiation.cc:299] Negotiation successful
0809 15:37:22.386246 (+ 123us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":2684,"thread_start_us":120,"threads_started":1}
I20260809 15:37:22.388247 16039 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:37:22.372155 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.89.195:40181 (local address 127.0.0.1:50668)
0809 15:37:22.376535 (+ 4380us) negotiation.cc:107] Waiting for socket to connect
0809 15:37:22.376588 (+ 53us) client_negotiation.cc:174] Beginning negotiation
0809 15:37:22.376709 (+ 121us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 15:37:22.384160 (+ 7451us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 15:37:22.384163 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 15:37:22.384180 (+ 17us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 15:37:22.384270 (+ 90us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:37:22.384275 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:37:22.385616 (+ 1341us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:37:22.385620 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:37:22.386752 (+ 1132us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:37:22.386762 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:37:22.387893 (+ 1131us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:37:22.387897 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:37:22.387982 (+ 85us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:37:22.388007 (+ 25us) client_negotiation.cc:769] Sending connection context
0809 15:37:22.388056 (+ 49us) client_negotiation.cc:240] Negotiation successful
0809 15:37:22.388120 (+ 64us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":4203,"thread_start_us":98,"threads_started":1}
I20260809 15:37:22.388962 16036 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:37:22.372298 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50668 (local address 127.3.89.195:40181)
0809 15:37:22.383498 (+ 11200us) server_negotiation.cc:206] Beginning negotiation
0809 15:37:22.383509 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0809 15:37:22.383532 (+ 23us) server_negotiation.cc:407] Connection header received
0809 15:37:22.383606 (+ 74us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 15:37:22.383611 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 15:37:22.383815 (+ 204us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 15:37:22.384013 (+ 198us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 15:37:22.384406 (+ 393us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:37:22.385480 (+ 1074us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:37:22.386945 (+ 1465us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:37:22.387736 (+ 791us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:37:22.388522 (+ 786us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:37:22.388663 (+ 141us) server_negotiation.cc:1035] Waiting for connection context
0809 15:37:22.388712 (+ 49us) server_negotiation.cc:299] Negotiation successful
0809 15:37:22.388811 (+ 99us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":11091,"thread_start_us":129,"threads_started":1}
I20260809 15:37:22.389866 15798 tablet_service.cc:1467] Processing CreateTablet for tablet e91aeafa0f39455a89663d79423d916e (DEFAULT_TABLE table=test-table [id=767cca8dc8554ff7bdab74753f64f9b5]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 15:37:22.391053 15798 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e91aeafa0f39455a89663d79423d916e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 15:37:22.391605 15950 tablet_service.cc:1467] Processing CreateTablet for tablet e91aeafa0f39455a89663d79423d916e (DEFAULT_TABLE table=test-table [id=767cca8dc8554ff7bdab74753f64f9b5]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 15:37:22.392570 15950 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e91aeafa0f39455a89663d79423d916e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 15:37:22.401037 16041 tablet_bootstrap.cc:492] T e91aeafa0f39455a89663d79423d916e P deeb678a8de14a2a80496d2867663ad6: Bootstrap starting.
I20260809 15:37:22.404709 16041 tablet_bootstrap.cc:654] T e91aeafa0f39455a89663d79423d916e P deeb678a8de14a2a80496d2867663ad6: Neither blocks nor log segments found. Creating new log.
I20260809 15:37:22.405661 16041 log.cc:824] T e91aeafa0f39455a89663d79423d916e P deeb678a8de14a2a80496d2867663ad6: Log is configured to *not* fsync() on all Append() calls
I20260809 15:37:22.407581 16035 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:37:22.375322 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.89.193:43925 (local address 127.0.0.1:36016)
0809 15:37:22.383138 (+ 7816us) negotiation.cc:107] Waiting for socket to connect
0809 15:37:22.383151 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0809 15:37:22.383240 (+ 89us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 15:37:22.389529 (+ 6289us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 15:37:22.389533 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 15:37:22.389554 (+ 21us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 15:37:22.389639 (+ 85us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:37:22.389645 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:37:22.393478 (+ 3833us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:37:22.393482 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:37:22.394701 (+ 1219us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:37:22.394712 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:37:22.407116 (+ 12404us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:37:22.407120 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:37:22.407236 (+ 116us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:37:22.407280 (+ 44us) client_negotiation.cc:769] Sending connection context
0809 15:37:22.407339 (+ 59us) client_negotiation.cc:240] Negotiation successful
0809 15:37:22.407424 (+ 85us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":7746,"thread_start_us":79,"threads_started":1}
I20260809 15:37:22.407819 16041 tablet_bootstrap.cc:492] T e91aeafa0f39455a89663d79423d916e P deeb678a8de14a2a80496d2867663ad6: No bootstrap required, opened a new log
I20260809 15:37:22.407994 16041 ts_tablet_manager.cc:1397] T e91aeafa0f39455a89663d79423d916e P deeb678a8de14a2a80496d2867663ad6: Time spent bootstrapping tablet: real 0.007s user 0.005s sys 0.000s
I20260809 15:37:22.408366 16040 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:37:22.375481 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36016 (local address 127.3.89.193:43925)
0809 15:37:22.388409 (+ 12928us) server_negotiation.cc:206] Beginning negotiation
0809 15:37:22.388416 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 15:37:22.388443 (+ 27us) server_negotiation.cc:407] Connection header received
0809 15:37:22.388517 (+ 74us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 15:37:22.388523 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 15:37:22.388712 (+ 189us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 15:37:22.389798 (+ 1086us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 15:37:22.390388 (+ 590us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:37:22.393330 (+ 2942us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:37:22.394885 (+ 1555us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:37:22.395723 (+ 838us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:37:22.407879 (+ 12156us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:37:22.408059 (+ 180us) server_negotiation.cc:1035] Waiting for connection context
0809 15:37:22.408117 (+ 58us) server_negotiation.cc:299] Negotiation successful
0809 15:37:22.408217 (+ 100us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":12814,"thread_start_us":74,"threads_started":1}
I20260809 15:37:22.411375 16042 tablet_bootstrap.cc:492] T e91aeafa0f39455a89663d79423d916e P fc960bf7e36c4f1a807a66332168b814: Bootstrap starting.
I20260809 15:37:22.413602 15653 tablet_service.cc:1467] Processing CreateTablet for tablet e91aeafa0f39455a89663d79423d916e (DEFAULT_TABLE table=test-table [id=767cca8dc8554ff7bdab74753f64f9b5]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 15:37:22.414151 16042 tablet_bootstrap.cc:654] T e91aeafa0f39455a89663d79423d916e P fc960bf7e36c4f1a807a66332168b814: Neither blocks nor log segments found. Creating new log.
I20260809 15:37:22.414737 15653 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e91aeafa0f39455a89663d79423d916e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 15:37:22.415045 16042 log.cc:824] T e91aeafa0f39455a89663d79423d916e P fc960bf7e36c4f1a807a66332168b814: Log is configured to *not* fsync() on all Append() calls
I20260809 15:37:22.420063 16041 raft_consensus.cc:348] T e91aeafa0f39455a89663d79423d916e P deeb678a8de14a2a80496d2867663ad6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fc960bf7e36c4f1a807a66332168b814" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 40181 } } peers { permanent_uuid: "deeb678a8de14a2a80496d2867663ad6" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 39039 } } peers { permanent_uuid: "428bbceec5914c4b8c7f64c5fd848ac6" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 43925 } }
I20260809 15:37:22.420488 16041 raft_consensus.cc:374] T e91aeafa0f39455a89663d79423d916e P deeb678a8de14a2a80496d2867663ad6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 15:37:22.420598 16041 raft_consensus.cc:729] T e91aeafa0f39455a89663d79423d916e P deeb678a8de14a2a80496d2867663ad6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: deeb678a8de14a2a80496d2867663ad6, State: Initialized, Role: FOLLOWER
I20260809 15:37:22.421172 16041 consensus_queue.cc:260] T e91aeafa0f39455a89663d79423d916e P deeb678a8de14a2a80496d2867663ad6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fc960bf7e36c4f1a807a66332168b814" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 40181 } } peers { permanent_uuid: "deeb678a8de14a2a80496d2867663ad6" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 39039 } } peers { permanent_uuid: "428bbceec5914c4b8c7f64c5fd848ac6" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 43925 } }
I20260809 15:37:22.425782 16041 ts_tablet_manager.cc:1428] T e91aeafa0f39455a89663d79423d916e P deeb678a8de14a2a80496d2867663ad6: Time spent starting tablet: real 0.018s user 0.006s sys 0.000s
I20260809 15:37:22.426204 15879 heartbeater.cc:503] Master 127.3.89.254:43043 was elected leader, sending a full tablet report...
W20260809 15:37:22.427172 15880 tablet.cc:2369] T e91aeafa0f39455a89663d79423d916e P deeb678a8de14a2a80496d2867663ad6: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260809 15:37:22.428993 16042 tablet_bootstrap.cc:492] T e91aeafa0f39455a89663d79423d916e P fc960bf7e36c4f1a807a66332168b814: No bootstrap required, opened a new log
I20260809 15:37:22.429212 16042 ts_tablet_manager.cc:1397] T e91aeafa0f39455a89663d79423d916e P fc960bf7e36c4f1a807a66332168b814: Time spent bootstrapping tablet: real 0.018s user 0.004s sys 0.000s
I20260809 15:37:22.433458 16042 raft_consensus.cc:348] T e91aeafa0f39455a89663d79423d916e P fc960bf7e36c4f1a807a66332168b814 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fc960bf7e36c4f1a807a66332168b814" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 40181 } } peers { permanent_uuid: "deeb678a8de14a2a80496d2867663ad6" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 39039 } } peers { permanent_uuid: "428bbceec5914c4b8c7f64c5fd848ac6" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 43925 } }
I20260809 15:37:22.433758 16042 raft_consensus.cc:374] T e91aeafa0f39455a89663d79423d916e P fc960bf7e36c4f1a807a66332168b814 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 15:37:22.433826 16042 raft_consensus.cc:729] T e91aeafa0f39455a89663d79423d916e P fc960bf7e36c4f1a807a66332168b814 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fc960bf7e36c4f1a807a66332168b814, State: Initialized, Role: FOLLOWER
I20260809 15:37:22.433930 16047 tablet_bootstrap.cc:492] T e91aeafa0f39455a89663d79423d916e P 428bbceec5914c4b8c7f64c5fd848ac6: Bootstrap starting.
I20260809 15:37:22.434378 16042 consensus_queue.cc:260] T e91aeafa0f39455a89663d79423d916e P fc960bf7e36c4f1a807a66332168b814 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fc960bf7e36c4f1a807a66332168b814" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 40181 } } peers { permanent_uuid: "deeb678a8de14a2a80496d2867663ad6" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 39039 } } peers { permanent_uuid: "428bbceec5914c4b8c7f64c5fd848ac6" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 43925 } }
I20260809 15:37:22.435474 16042 ts_tablet_manager.cc:1428] T e91aeafa0f39455a89663d79423d916e P fc960bf7e36c4f1a807a66332168b814: Time spent starting tablet: real 0.006s user 0.007s sys 0.000s
I20260809 15:37:22.436061 16021 heartbeater.cc:503] Master 127.3.89.254:43043 was elected leader, sending a full tablet report...
I20260809 15:37:22.436460 16047 tablet_bootstrap.cc:654] T e91aeafa0f39455a89663d79423d916e P 428bbceec5914c4b8c7f64c5fd848ac6: Neither blocks nor log segments found. Creating new log.
I20260809 15:37:22.437410 16047 log.cc:824] T e91aeafa0f39455a89663d79423d916e P 428bbceec5914c4b8c7f64c5fd848ac6: Log is configured to *not* fsync() on all Append() calls
I20260809 15:37:22.439711 16047 tablet_bootstrap.cc:492] T e91aeafa0f39455a89663d79423d916e P 428bbceec5914c4b8c7f64c5fd848ac6: No bootstrap required, opened a new log
I20260809 15:37:22.439910 16047 ts_tablet_manager.cc:1397] T e91aeafa0f39455a89663d79423d916e P 428bbceec5914c4b8c7f64c5fd848ac6: Time spent bootstrapping tablet: real 0.006s user 0.005s sys 0.000s
I20260809 15:37:22.444423 16047 raft_consensus.cc:348] T e91aeafa0f39455a89663d79423d916e P 428bbceec5914c4b8c7f64c5fd848ac6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fc960bf7e36c4f1a807a66332168b814" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 40181 } } peers { permanent_uuid: "deeb678a8de14a2a80496d2867663ad6" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 39039 } } peers { permanent_uuid: "428bbceec5914c4b8c7f64c5fd848ac6" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 43925 } }
I20260809 15:37:22.444693 16047 raft_consensus.cc:374] T e91aeafa0f39455a89663d79423d916e P 428bbceec5914c4b8c7f64c5fd848ac6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 15:37:22.444758 16047 raft_consensus.cc:729] T e91aeafa0f39455a89663d79423d916e P 428bbceec5914c4b8c7f64c5fd848ac6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 428bbceec5914c4b8c7f64c5fd848ac6, State: Initialized, Role: FOLLOWER
I20260809 15:37:22.445204 16047 consensus_queue.cc:260] T e91aeafa0f39455a89663d79423d916e P 428bbceec5914c4b8c7f64c5fd848ac6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fc960bf7e36c4f1a807a66332168b814" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 40181 } } peers { permanent_uuid: "deeb678a8de14a2a80496d2867663ad6" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 39039 } } peers { permanent_uuid: "428bbceec5914c4b8c7f64c5fd848ac6" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 43925 } }
I20260809 15:37:22.445976 15734 heartbeater.cc:503] Master 127.3.89.254:43043 was elected leader, sending a full tablet report...
I20260809 15:37:22.446250 16047 ts_tablet_manager.cc:1428] T e91aeafa0f39455a89663d79423d916e P 428bbceec5914c4b8c7f64c5fd848ac6: Time spent starting tablet: real 0.006s user 0.006s sys 0.000s
W20260809 15:37:22.546097 16023 tablet.cc:2369] T e91aeafa0f39455a89663d79423d916e P fc960bf7e36c4f1a807a66332168b814: Can't schedule compaction. Clean time has not been advanced past its initial value.
Aug 09 15:37:22 dist-test-slave-2w3w krb5kdc[15003](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786289841, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.89.254@KRBTEST.COM for HTTP/127.3.89.212@KRBTEST.COM
W20260809 15:37:22.623080 15735 tablet.cc:2369] T e91aeafa0f39455a89663d79423d916e P 428bbceec5914c4b8c7f64c5fd848ac6: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260809 15:37:22.677268 16048 raft_consensus.cc:482] T e91aeafa0f39455a89663d79423d916e P fc960bf7e36c4f1a807a66332168b814 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 15:37:22.677532 16048 raft_consensus.cc:504] T e91aeafa0f39455a89663d79423d916e P fc960bf7e36c4f1a807a66332168b814 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fc960bf7e36c4f1a807a66332168b814" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 40181 } } peers { permanent_uuid: "deeb678a8de14a2a80496d2867663ad6" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 39039 } } peers { permanent_uuid: "428bbceec5914c4b8c7f64c5fd848ac6" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 43925 } }
I20260809 15:37:22.679045 16048 leader_election.cc:288] T e91aeafa0f39455a89663d79423d916e P fc960bf7e36c4f1a807a66332168b814 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers deeb678a8de14a2a80496d2867663ad6 (127.3.89.194:39039), 428bbceec5914c4b8c7f64c5fd848ac6 (127.3.89.193:43925)
I20260809 15:37:22.685287 16024 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:37:22.679270 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.89.194:39039 (local address 127.3.89.195:48407)
0809 15:37:22.679522 (+ 252us) negotiation.cc:107] Waiting for socket to connect
0809 15:37:22.679540 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0809 15:37:22.679664 (+ 124us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 15:37:22.680886 (+ 1222us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 15:37:22.680890 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 15:37:22.680909 (+ 19us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 15:37:22.681032 (+ 123us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:37:22.681039 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:37:22.682091 (+ 1052us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:37:22.682094 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:37:22.683406 (+ 1312us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:37:22.683420 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:37:22.684821 (+ 1401us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:37:22.684825 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:37:22.684952 (+ 127us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:37:22.684977 (+ 25us) client_negotiation.cc:769] Sending connection context
0809 15:37:22.685054 (+ 77us) client_negotiation.cc:240] Negotiation successful
0809 15:37:22.685130 (+ 76us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":161}
I20260809 15:37:22.686048 16038 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:37:22.680364 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.89.195:48407 (local address 127.3.89.194:39039)
0809 15:37:22.680523 (+ 159us) server_negotiation.cc:206] Beginning negotiation
0809 15:37:22.680528 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 15:37:22.680544 (+ 16us) server_negotiation.cc:407] Connection header received
0809 15:37:22.680599 (+ 55us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 15:37:22.680602 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 15:37:22.680654 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 15:37:22.680734 (+ 80us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 15:37:22.681176 (+ 442us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:37:22.681943 (+ 767us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:37:22.683962 (+ 2019us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:37:22.684583 (+ 621us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:37:22.684702 (+ 119us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:37:22.684756 (+ 54us) server_negotiation.cc:1035] Waiting for connection context
0809 15:37:22.685822 (+ 1066us) server_negotiation.cc:299] Negotiation successful
0809 15:37:22.685905 (+ 83us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":44}
I20260809 15:37:22.686676 15831 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e91aeafa0f39455a89663d79423d916e" candidate_uuid: "fc960bf7e36c4f1a807a66332168b814" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "deeb678a8de14a2a80496d2867663ad6" is_pre_election: true
I20260809 15:37:22.687036 15831 raft_consensus.cc:2454] T e91aeafa0f39455a89663d79423d916e P deeb678a8de14a2a80496d2867663ad6 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fc960bf7e36c4f1a807a66332168b814 in term 0.
I20260809 15:37:22.687620 15903 leader_election.cc:302] T e91aeafa0f39455a89663d79423d916e P fc960bf7e36c4f1a807a66332168b814 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: deeb678a8de14a2a80496d2867663ad6, fc960bf7e36c4f1a807a66332168b814; no voters:
I20260809 15:37:22.687922 16048 raft_consensus.cc:2781] T e91aeafa0f39455a89663d79423d916e P fc960bf7e36c4f1a807a66332168b814 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260809 15:37:22.688011 16048 raft_consensus.cc:482] T e91aeafa0f39455a89663d79423d916e P fc960bf7e36c4f1a807a66332168b814 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 15:37:22.688059 16048 raft_consensus.cc:3037] T e91aeafa0f39455a89663d79423d916e P fc960bf7e36c4f1a807a66332168b814 [term 0 FOLLOWER]: Advancing to term 1
I20260809 15:37:22.689661 16048 raft_consensus.cc:504] T e91aeafa0f39455a89663d79423d916e P fc960bf7e36c4f1a807a66332168b814 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fc960bf7e36c4f1a807a66332168b814" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 40181 } } peers { permanent_uuid: "deeb678a8de14a2a80496d2867663ad6" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 39039 } } peers { permanent_uuid: "428bbceec5914c4b8c7f64c5fd848ac6" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 43925 } }
I20260809 15:37:22.690032 16048 leader_election.cc:288] T e91aeafa0f39455a89663d79423d916e P fc960bf7e36c4f1a807a66332168b814 [CANDIDATE]: Term 1 election: Requested vote from peers deeb678a8de14a2a80496d2867663ad6 (127.3.89.194:39039), 428bbceec5914c4b8c7f64c5fd848ac6 (127.3.89.193:43925)
I20260809 15:37:22.690598 15831 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e91aeafa0f39455a89663d79423d916e" candidate_uuid: "fc960bf7e36c4f1a807a66332168b814" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "deeb678a8de14a2a80496d2867663ad6"
I20260809 15:37:22.690727 15831 raft_consensus.cc:3037] T e91aeafa0f39455a89663d79423d916e P deeb678a8de14a2a80496d2867663ad6 [term 0 FOLLOWER]: Advancing to term 1
I20260809 15:37:22.691358 16040 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:37:22.680516 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.89.195:52329 (local address 127.3.89.193:43925)
0809 15:37:22.680641 (+ 125us) server_negotiation.cc:206] Beginning negotiation
0809 15:37:22.680646 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 15:37:22.683539 (+ 2893us) server_negotiation.cc:407] Connection header received
0809 15:37:22.683601 (+ 62us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 15:37:22.683603 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 15:37:22.683658 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 15:37:22.683731 (+ 73us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 15:37:22.686941 (+ 3210us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:37:22.687694 (+ 753us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:37:22.689555 (+ 1861us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:37:22.690197 (+ 642us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:37:22.690320 (+ 123us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:37:22.690390 (+ 70us) server_negotiation.cc:1035] Waiting for connection context
0809 15:37:22.691081 (+ 691us) server_negotiation.cc:299] Negotiation successful
0809 15:37:22.691183 (+ 102us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":49}
I20260809 15:37:22.692026 15831 raft_consensus.cc:2454] T e91aeafa0f39455a89663d79423d916e P deeb678a8de14a2a80496d2867663ad6 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fc960bf7e36c4f1a807a66332168b814 in term 1.
I20260809 15:37:22.692059 16053 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:37:22.679585 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.89.193:43925 (local address 127.3.89.195:52329)
0809 15:37:22.683126 (+ 3541us) negotiation.cc:107] Waiting for socket to connect
0809 15:37:22.683142 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0809 15:37:22.683241 (+ 99us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 15:37:22.686564 (+ 3323us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 15:37:22.686568 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 15:37:22.686589 (+ 21us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 15:37:22.686712 (+ 123us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:37:22.686718 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:37:22.687951 (+ 1233us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:37:22.687954 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:37:22.689368 (+ 1414us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:37:22.689378 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:37:22.690637 (+ 1259us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:37:22.690643 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:37:22.690874 (+ 231us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:37:22.690911 (+ 37us) client_negotiation.cc:769] Sending connection context
0809 15:37:22.691682 (+ 771us) client_negotiation.cc:240] Negotiation successful
0809 15:37:22.691876 (+ 194us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":3472,"thread_start_us":141,"threads_started":1}
I20260809 15:37:22.692363 15903 leader_election.cc:302] T e91aeafa0f39455a89663d79423d916e P fc960bf7e36c4f1a807a66332168b814 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: deeb678a8de14a2a80496d2867663ad6, fc960bf7e36c4f1a807a66332168b814; no voters:
I20260809 15:37:22.692531 16048 raft_consensus.cc:2781] T e91aeafa0f39455a89663d79423d916e P fc960bf7e36c4f1a807a66332168b814 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 15:37:22.692739 16048 raft_consensus.cc:686] T e91aeafa0f39455a89663d79423d916e P fc960bf7e36c4f1a807a66332168b814 [term 1 LEADER]: Becoming Leader. State: Replica: fc960bf7e36c4f1a807a66332168b814, State: Running, Role: LEADER
I20260809 15:37:22.692873 15682 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e91aeafa0f39455a89663d79423d916e" candidate_uuid: "fc960bf7e36c4f1a807a66332168b814" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "428bbceec5914c4b8c7f64c5fd848ac6" is_pre_election: true
I20260809 15:37:22.692898 15680 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e91aeafa0f39455a89663d79423d916e" candidate_uuid: "fc960bf7e36c4f1a807a66332168b814" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "428bbceec5914c4b8c7f64c5fd848ac6"
I20260809 15:37:22.693089 16048 consensus_queue.cc:237] T e91aeafa0f39455a89663d79423d916e P fc960bf7e36c4f1a807a66332168b814 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fc960bf7e36c4f1a807a66332168b814" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 40181 } } peers { permanent_uuid: "deeb678a8de14a2a80496d2867663ad6" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 39039 } } peers { permanent_uuid: "428bbceec5914c4b8c7f64c5fd848ac6" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 43925 } }
I20260809 15:37:22.693137 15682 raft_consensus.cc:2454] T e91aeafa0f39455a89663d79423d916e P 428bbceec5914c4b8c7f64c5fd848ac6 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fc960bf7e36c4f1a807a66332168b814 in term 0.
I20260809 15:37:22.696283 15509 catalog_manager.cc:5495] T e91aeafa0f39455a89663d79423d916e P fc960bf7e36c4f1a807a66332168b814 reported cstate change: term changed from 0 to 1, leader changed from <none> to fc960bf7e36c4f1a807a66332168b814 (127.3.89.195). New cstate: current_term: 1 leader_uuid: "fc960bf7e36c4f1a807a66332168b814" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fc960bf7e36c4f1a807a66332168b814" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 40181 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "deeb678a8de14a2a80496d2867663ad6" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 39039 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "428bbceec5914c4b8c7f64c5fd848ac6" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 43925 } health_report { overall_health: UNKNOWN } } }
I20260809 15:37:22.763842 16036 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:37:22.758316 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50680 (local address 127.3.89.195:40181)
0809 15:37:22.758451 (+ 135us) server_negotiation.cc:206] Beginning negotiation
0809 15:37:22.758456 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 15:37:22.758473 (+ 17us) server_negotiation.cc:407] Connection header received
0809 15:37:22.758663 (+ 190us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 15:37:22.758667 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 15:37:22.758748 (+ 81us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 15:37:22.758871 (+ 123us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 15:37:22.759307 (+ 436us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:37:22.760113 (+ 806us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:37:22.761445 (+ 1332us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:37:22.761770 (+ 325us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:37:22.761887 (+ 117us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:37:22.762339 (+ 452us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 15:37:22.763024 (+ 685us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 15:37:22.763154 (+ 130us) server_negotiation.cc:1035] Waiting for connection context
0809 15:37:22.763626 (+ 472us) server_negotiation.cc:299] Negotiation successful
0809 15:37:22.763695 (+ 69us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":42}
I20260809 15:37:22.771016 15680 raft_consensus.cc:3037] T e91aeafa0f39455a89663d79423d916e P 428bbceec5914c4b8c7f64c5fd848ac6 [term 0 FOLLOWER]: Advancing to term 1
I20260809 15:37:22.772248 15831 raft_consensus.cc:1261] T e91aeafa0f39455a89663d79423d916e P deeb678a8de14a2a80496d2867663ad6 [term 1 FOLLOWER]: Refusing update from remote peer fc960bf7e36c4f1a807a66332168b814: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 15:37:22.772580 15680 raft_consensus.cc:1261] T e91aeafa0f39455a89663d79423d916e P 428bbceec5914c4b8c7f64c5fd848ac6 [term 1 FOLLOWER]: Refusing update from remote peer fc960bf7e36c4f1a807a66332168b814: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 15:37:22.772971 16054 consensus_queue.cc:1035] T e91aeafa0f39455a89663d79423d916e P fc960bf7e36c4f1a807a66332168b814 [LEADER]: Connected to new peer: Peer: permanent_uuid: "deeb678a8de14a2a80496d2867663ad6" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 39039 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 15:37:22.773134 16048 consensus_queue.cc:1035] T e91aeafa0f39455a89663d79423d916e P fc960bf7e36c4f1a807a66332168b814 [LEADER]: Connected to new peer: Peer: permanent_uuid: "428bbceec5914c4b8c7f64c5fd848ac6" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 43925 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 15:37:22.784080 16057 mvcc.cc:205] Tried to move back new op lower bound from 7316643195981492224 to 7316643195673808896. Current Snapshot: MvccSnapshot[applied={T|T < 7316643195981492224}]
I20260809 15:37:22.787011 16059 mvcc.cc:205] Tried to move back new op lower bound from 7316643195981492224 to 7316643195673808896. Current Snapshot: MvccSnapshot[applied={T|T < 7316643195981492224}]
I20260809 15:37:22.787506 16058 mvcc.cc:205] Tried to move back new op lower bound from 7316643195981492224 to 7316643195673808896. Current Snapshot: MvccSnapshot[applied={T|T < 7316643195981492224}]
I20260809 15:37:22.796788 15508 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:45704:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 15:37:22.796972 15508 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:45704:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 15:37:22.800004 15508 catalog_manager.cc:5782] T 00000000000000000000000000000000 P cfa3b33d5ca74124b71aac50c9fe2751: Sending DeleteTablet for 3 replicas of tablet e91aeafa0f39455a89663d79423d916e
I20260809 15:37:22.800930 15798 tablet_service.cc:1514] Processing DeleteTablet for tablet e91aeafa0f39455a89663d79423d916e with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 15:37:22 UTC) from {username='kudu', principal='kudu/127.3.89.254@KRBTEST.COM'} at 127.0.0.1:51042
I20260809 15:37:22.801857 15653 tablet_service.cc:1514] Processing DeleteTablet for tablet e91aeafa0f39455a89663d79423d916e with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 15:37:22 UTC) from {username='kudu', principal='kudu/127.3.89.254@KRBTEST.COM'} at 127.0.0.1:36016
I20260809 15:37:22.802506 15950 tablet_service.cc:1514] Processing DeleteTablet for tablet e91aeafa0f39455a89663d79423d916e with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 15:37:22 UTC) from {username='kudu', principal='kudu/127.3.89.254@KRBTEST.COM'} at 127.0.0.1:50668
I20260809 15:37:22.803527 3431 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskm314YR/build/debug/bin/kudu with pid 15597
I20260809 15:37:22.813651 16068 tablet_replica.cc:330] T e91aeafa0f39455a89663d79423d916e P fc960bf7e36c4f1a807a66332168b814: stopping tablet replica
I20260809 15:37:22.814033 16068 raft_consensus.cc:2229] T e91aeafa0f39455a89663d79423d916e P fc960bf7e36c4f1a807a66332168b814 [term 1 LEADER]: Raft consensus shutting down.
I20260809 15:37:22.814459 16068 raft_consensus.cc:2258] T e91aeafa0f39455a89663d79423d916e P fc960bf7e36c4f1a807a66332168b814 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 15:37:22.814829 16066 tablet_replica.cc:330] T e91aeafa0f39455a89663d79423d916e P deeb678a8de14a2a80496d2867663ad6: stopping tablet replica
I20260809 15:37:22.816311 16068 ts_tablet_manager.cc:1905] T e91aeafa0f39455a89663d79423d916e P fc960bf7e36c4f1a807a66332168b814: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 15:37:22.819236 16066 raft_consensus.cc:2229] T e91aeafa0f39455a89663d79423d916e P deeb678a8de14a2a80496d2867663ad6 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 15:37:22.819615 16066 raft_consensus.cc:2258] T e91aeafa0f39455a89663d79423d916e P deeb678a8de14a2a80496d2867663ad6 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 15:37:22.820062 16068 ts_tablet_manager.cc:1918] T e91aeafa0f39455a89663d79423d916e P fc960bf7e36c4f1a807a66332168b814: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 15:37:22.820173 16068 log.cc:1196] T e91aeafa0f39455a89663d79423d916e P fc960bf7e36c4f1a807a66332168b814: Deleting WAL directory at /tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786289732823074-3431-0/minicluster-data/ts-2/wal/wals/e91aeafa0f39455a89663d79423d916e
I20260809 15:37:22.820490 16068 ts_tablet_manager.cc:1939] T e91aeafa0f39455a89663d79423d916e P fc960bf7e36c4f1a807a66332168b814: Deleting consensus metadata
I20260809 15:37:22.821743 15493 catalog_manager.cc:4841] TS fc960bf7e36c4f1a807a66332168b814 (127.3.89.195:40181): tablet e91aeafa0f39455a89663d79423d916e (table test-table [id=767cca8dc8554ff7bdab74753f64f9b5]) successfully deleted
I20260809 15:37:22.823119 16066 ts_tablet_manager.cc:1905] T e91aeafa0f39455a89663d79423d916e P deeb678a8de14a2a80496d2867663ad6: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 15:37:22.825698 16066 ts_tablet_manager.cc:1918] T e91aeafa0f39455a89663d79423d916e P deeb678a8de14a2a80496d2867663ad6: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 15:37:22.825801 16066 log.cc:1196] T e91aeafa0f39455a89663d79423d916e P deeb678a8de14a2a80496d2867663ad6: Deleting WAL directory at /tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786289732823074-3431-0/minicluster-data/ts-1/wal/wals/e91aeafa0f39455a89663d79423d916e
I20260809 15:37:22.826135 16066 ts_tablet_manager.cc:1939] T e91aeafa0f39455a89663d79423d916e P deeb678a8de14a2a80496d2867663ad6: Deleting consensus metadata
W20260809 15:37:22.826660 15493 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.3.89.193:43925 (error 108)
I20260809 15:37:22.827658 3431 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskm314YR/build/debug/bin/kudu with pid 15742
I20260809 15:37:22.835975 16035 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:37:22.835600 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.89.193:43925 (local address 127.0.0.1:36030)
0809 15:37:22.835795 (+ 195us) negotiation.cc:107] Waiting for socket to connect
0809 15:37:22.835872 (+ 77us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.89.193:43925: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":115}
W20260809 15:37:22.836175 15493 catalog_manager.cc:4584] TS 428bbceec5914c4b8c7f64c5fd848ac6 (127.3.89.193:43925): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet e91aeafa0f39455a89663d79423d916e: Network error: Client connection negotiation failed: client connection to 127.3.89.193:43925: connect: Connection refused (error 111)
W20260809 15:37:22.839371 16035 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 15:37:22.838751 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.89.194:39039 (local address 127.0.0.1:51050)
0809 15:37:22.838897 (+ 146us) negotiation.cc:107] Waiting for socket to connect
0809 15:37:22.838910 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0809 15:37:22.839058 (+ 148us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 15:37:22.839264 (+ 206us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.89.194:39039: BlockingRecv error: recv error from unknown peer: Transport endpoint is not connected (error 107)
Metrics: {"client-negotiator.queue_time_us":77}
I20260809 15:37:22.839990 3431 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskm314YR/build/debug/bin/kudu with pid 15884
I20260809 15:37:22.847954 3431 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskm314YR/build/debug/bin/kudu with pid 15477
I20260809 15:37:22.855847 3431 mini_ranger_kms.cc:60] Stopping Ranger KMS...
I20260809 15:37:23.200870 3431 mini_ranger_kms.cc:62] Stopped Ranger KMS
I20260809 15:37:23.201036 3431 mini_ranger.cc:66] Stopping Ranger...
I20260809 15:37:23.545900 3431 mini_ranger.cc:68] Stopped Ranger
2026-08-09 15:37:23.546 UTC [15035] LOG: received smart shutdown request
2026-08-09 15:37:23.550 UTC [15035] LOG: background worker "logical replication launcher" (PID 15044) exited with exit code 1
2026-08-09 15:37:23.550 UTC [15039] LOG: shutting down
2026-08-09 15:37:23.716 UTC [15035] LOG: database system is shut down
2026-08-09T15:37:23Z chronyd exiting
[ OK ] SecurityITest.TestEncryptionWithKMSIntegration (47461 ms)
[ RUN ] SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers
Loading random data
Initializing database '/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786289732823074-3431-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 15:37:23 dist-test-slave-2w3w krb5kdc[16078](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 15:37:23 dist-test-slave-2w3w krb5kdc[16078](info): set up 2 sockets
Aug 09 15:37:23 dist-test-slave-2w3w krb5kdc[16078](info): commencing operation
krb5kdc: starting...
W20260809 15:37:24.976682 3431 mini_kdc.cc:121] Time spent starting KDC: real 1.113s user 0.004s sys 0.004s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786289732823074-3431-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786289732823074-3431-0/krb5kdc/test-user.keytab.
Aug 09 15:37:25 dist-test-slave-2w3w krb5kdc[16078](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786289845, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T15:37:25Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T15:37:25Z Disabled control of system clock
WARNING: no policy specified for rangeradmin/127.3.89.212@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangeradmin/127.3.89.212@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangeradmin/127.3.89.212@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786289732823074-3431-0/krb5kdc/rangeradmin_127.3.89.212@KRBTEST.COM.keytab.
Entry for principal rangeradmin/127.3.89.212@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786289732823074-3431-0/krb5kdc/rangeradmin_127.3.89.212@KRBTEST.COM.keytab.
WARNING: no policy specified for rangerlookup/127.3.89.212@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangerlookup/127.3.89.212@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangerlookup/127.3.89.212@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786289732823074-3431-0/krb5kdc/rangerlookup_127.3.89.212@KRBTEST.COM.keytab.
Entry for principal rangerlookup/127.3.89.212@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786289732823074-3431-0/krb5kdc/rangerlookup_127.3.89.212@KRBTEST.COM.keytab.
WARNING: no policy specified for HTTP/127.3.89.212@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "HTTP/127.3.89.212@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal HTTP/127.3.89.212@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786289732823074-3431-0/krb5kdc/HTTP_127.3.89.212@KRBTEST.COM.keytab.
Entry for principal HTTP/127.3.89.212@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786289732823074-3431-0/krb5kdc/HTTP_127.3.89.212@KRBTEST.COM.keytab.
I20260809 15:37:25.153860 3431 mini_postgres.cc:62] Running initdb...
The files belonging to this database system will be owned by user "slave".
This user must also own the server process.
The database cluster will be initialized with locale "C".
The default database encoding has accordingly been set to "SQL_ASCII".
The default text search configuration will be set to "english".
Data page checksums are disabled.
creating directory /tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786289732823074-3431-0/minicluster-data/postgres ... ok
creating subdirectories ... ok
selecting dynamic shared memory implementation ... posix
selecting default max_connections ... 100
selecting default shared_buffers ... 128MB
selecting default time zone ... Etc/UTC
creating configuration files ... ok
running bootstrap script ... ok
performing post-bootstrap initialization ... ok
syncing data to disk ... ok
initdb: warning: enabling "trust" authentication for local connections
You can change this by editing pg_hba.conf or using the option -A, or
--auth-local and --auth-host, the next time you run initdb.
Success. You can now start the database server using:
/tmp/dist-test-taskm314YR/build/debug/bin/postgres/pg_ctl -D /tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786289732823074-3431-0/minicluster-data/postgres -l logfile start
2026-08-09 15:37:27.569 UTC [16109] LOG: starting PostgreSQL 12.2 on x86_64-pc-linux-gnu, compiled by cc (GCC) 8.3.1 20190311 (Red Hat 8.3.1-3), 64-bit
2026-08-09 15:37:27.569 UTC [16109] LOG: listening on IPv4 address "127.3.89.212", port 44657
2026-08-09 15:37:27.572 UTC [16109] LOG: listening on Unix socket "/tmp/.s.PGSQL.44657"
2026-08-09 15:37:27.594 UTC [16112] LOG: database system was shut down at 2026-08-09 15:37:26 UTC
2026-08-09 15:37:27.598 UTC [16109] LOG: database system is ready to accept connections
I20260809 15:37:28.613066 3431 mini_postgres.cc:96] Postgres bound to 44657
2026-08-09 15:37:28.620 UTC [16120] FATAL: database "slave" does not exist
127.3.89.212:44657 - accepting connections
I20260809 15:37:28.621516 3431 mini_ranger.cc:161] Starting Ranger...
I20260809 15:37:28.637281 3431 mini_ranger.cc:84] Created miniranger Postgres user
I20260809 15:37:29.108301 3431 mini_ranger.cc:87] Created ranger Postgres database
I20260809 15:37:29.108402 3431 mini_ranger.cc:178] Starting Ranger out of /tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786289732823074-3431-0/minicluster-data/ranger-admin
2026-08-09 15:37:29,379 [I] DB FLAVOR :POSTGRES
2026-08-09 15:37:29,380 [I] --------- Verifying Ranger DB connection ---------
2026-08-09 15:37:29,380 [I] Checking connection..
2026-08-09 15:37:29,380 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskm314YR/build/debug/bin/postgresql.jar:/tmp/dist-test-taskm314YR/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.3.89.212:44657/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select 1;"
2026-08-09 15:37:29,671 [I] Checking connection passed.
2026-08-09 15:37:29,671 [I] --------- Verifying version history table ---------
2026-08-09 15:37:29,671 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskm314YR/build/debug/bin/postgresql.jar:/tmp/dist-test-taskm314YR/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.3.89.212:44657/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-08-09 15:37:29,938 [I] Table x_db_version_h does not exist in database ranger
2026-08-09 15:37:29,938 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskm314YR/build/debug/bin/postgresql.jar:/tmp/dist-test-taskm314YR/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.3.89.212:44657/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-08-09 15:37:30,196 [I] Table x_db_version_h does not exist in database ranger
2026-08-09 15:37:30,196 [I] Importing x_db_version_h table schema to database ranger from file: create_dbversion_catalog.sql
2026-08-09 15:37:30,197 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskm314YR/build/debug/bin/postgresql.jar:/tmp/dist-test-taskm314YR/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.3.89.212:44657/ranger -u miniranger -p '********' -noheader -trim -c \; -input /tmp/dist-test-taskm314YR/thirdparty/src/ranger-2.1.0-admin/db/postgres/create_dbversion_catalog.sql
2026-08-09 15:37:30.434 UTC [16187] WARNING: there is no transaction in progress
2026-08-09 15:37:30,441 [I] create_dbversion_catalog.sql file imported successfully
2026-08-09 15:37:30,442 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskm314YR/build/debug/bin/postgresql.jar:/tmp/dist-test-taskm314YR/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.3.89.212:44657/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-08-09 15:37:30,694 [I] Table x_db_version_h already exists in database 'ranger'
2026-08-09 15:37:30,694 [I] --------- Importing Ranger Core DB Schema ---------
2026-08-09 15:37:30,694 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskm314YR/build/debug/bin/postgresql.jar:/tmp/dist-test-taskm314YR/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.3.89.212:44657/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'CORE_DB_SCHEMA' and active = 'Y';"
2026-08-09 15:37:30,940 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskm314YR/build/debug/bin/postgresql.jar:/tmp/dist-test-taskm314YR/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.3.89.212:44657/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'CORE_DB_SCHEMA' and active = 'N';"
2026-08-09 15:37:31,176 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskm314YR/build/debug/bin/postgresql.jar:/tmp/dist-test-taskm314YR/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.3.89.212:44657/ranger -u miniranger -p '********' -noheader -trim -c \; -query "insert into x_db_version_h (version, inst_at, inst_by, updated_at, updated_by,active) values ('CORE_DB_SCHEMA', current_timestamp, 'Ranger 2.1.0', current_timestamp, 'dist-test-slave-2w3w.c.gcp-upstream.internal','N') ;"
2026-08-09 15:37:31,423 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskm314YR/build/debug/bin/postgresql.jar:/tmp/dist-test-taskm314YR/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.3.89.212:44657/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_portal_user') as temp;"
2026-08-09 15:37:31,684 [I] Table x_portal_user does not exist in database ranger
2026-08-09 15:37:31,684 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskm314YR/build/debug/bin/postgresql.jar:/tmp/dist-test-taskm314YR/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.3.89.212:44657/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_policy_ref_group') as temp;"
2026-08-09 15:37:31,934 [I] Table x_policy_ref_group does not exist in database ranger
2026-08-09 15:37:31,934 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskm314YR/build/debug/bin/postgresql.jar:/tmp/dist-test-taskm314YR/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.3.89.212:44657/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and active = 'Y';"
2026-08-09 15:37:32,179 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskm314YR/build/debug/bin/postgresql.jar:/tmp/dist-test-taskm314YR/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.3.89.212:44657/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'JAVA_PATCHES' and active = 'Y';"
2026-08-09 15:37:32,430 [I] Importing DB schema to database ranger from file: ranger_core_db_postgres.sql
2026-08-09 15:37:32,430 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskm314YR/build/debug/bin/postgresql.jar:/tmp/dist-test-taskm314YR/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.3.89.212:44657/ranger -u miniranger -p '********' -noheader -trim -c \; -input /tmp/dist-test-taskm314YR/thirdparty/src/ranger-2.1.0-admin/db/postgres/optimized/current/ranger_core_db_postgres.sql
2026-08-09 15:37:33.386 UTC [16301] WARNING: there is no transaction in progress
2026-08-09 15:37:33.402 UTC [16301] WARNING: there is no transaction in progress
2026-08-09 15:37:33.418 UTC [16301] WARNING: there is no transaction in progress
2026-08-09 15:37:33.432 UTC [16301] WARNING: there is no transaction in progress
2026-08-09 15:37:33.447 UTC [16301] WARNING: there is no transaction in progress
2026-08-09 15:37:33.464 UTC [16301] WARNING: there is no transaction in progress
2026-08-09 15:37:33.516 UTC [16301] WARNING: there is no transaction in progress
2026-08-09 15:37:33.522 UTC [16301] WARNING: there is no transaction in progress
2026-08-09 15:37:33.536 UTC [16301] WARNING: there is no transaction in progress
2026-08-09 15:37:33.546 UTC [16301] WARNING: there is no transaction in progress
2026-08-09 15:37:33.556 UTC [16301] WARNING: there is no transaction in progress
2026-08-09 15:37:33.569 UTC [16301] WARNING: there is no transaction in progress
2026-08-09 15:37:33.578 UTC [16301] WARNING: there is no transaction in progress
2026-08-09 15:37:33.584 UTC [16301] WARNING: there is no transaction in progress
2026-08-09 15:37:33.591 UTC [16301] WARNING: there is no transaction in progress
2026-08-09 15:37:33.597 UTC [16301] WARNING: there is no transaction in progress
2026-08-09 15:37:34,217 [I] ranger_core_db_postgres.sql file imported successfully
2026-08-09 15:37:34,217 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskm314YR/build/debug/bin/postgresql.jar:/tmp/dist-test-taskm314YR/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.3.89.212:44657/ranger -u miniranger -p '********' -noheader -trim -c \; -query "update x_db_version_h set inst_by='Ranger 2.1.0' where active='Y' and updated_by='localhost';"
2026-08-09 15:37:34,479 [I] Patches status entries updated from base ranger version to current installed ranger version:Ranger 2.1.0
2026-08-09 15:37:34,479 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskm314YR/build/debug/bin/postgresql.jar:/tmp/dist-test-taskm314YR/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.3.89.212:44657/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_portal_user') as temp;"
2026-08-09 15:37:34,747 [I] Table x_portal_user already exists in database 'ranger'
2026-08-09 15:37:34,747 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskm314YR/build/debug/bin/postgresql.jar:/tmp/dist-test-taskm314YR/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.3.89.212:44657/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_policy_ref_group') as temp;"
2026-08-09 15:37:34,997 [I] Table x_policy_ref_group already exists in database 'ranger'
2026-08-09 15:37:34,997 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskm314YR/build/debug/bin/postgresql.jar:/tmp/dist-test-taskm314YR/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.3.89.212:44657/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and active = 'Y';"
2026-08-09 15:37:35,234 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskm314YR/build/debug/bin/postgresql.jar:/tmp/dist-test-taskm314YR/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.3.89.212:44657/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'JAVA_PATCHES' and active = 'Y';"
2026-08-09 15:37:35,494 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskm314YR/build/debug/bin/postgresql.jar:/tmp/dist-test-taskm314YR/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.3.89.212:44657/ranger -u miniranger -p '********' -noheader -trim -c \; -query "update x_db_version_h set active='Y' where version='CORE_DB_SCHEMA' and active='N' and updated_by='dist-test-slave-2w3w.c.gcp-upstream.internal';"
2026-08-09 15:37:35,745 [I] CORE_DB_SCHEMA import status has been updated
2026-08-09 15:37:35,746 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskm314YR/build/debug/bin/postgresql.jar:/tmp/dist-test-taskm314YR/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.3.89.212:44657/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and inst_by = 'Ranger 2.1.0' and active = 'Y';"
2026-08-09 15:37:35,980 [I] DB_PATCHES have already been applied
I20260809 15:37:35.985823 3431 mini_ranger.cc:191] Using Ranger class path: /tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786289732823074-3431-0/minicluster-data/ranger-admin:/tmp/dist-test-taskm314YR/thirdparty/src/ranger-2.1.0-admin/ews/lib/*:/usr/lib/jvm/java-1.8.0-openjdk-amd64/lib/*:/tmp/dist-test-taskm314YR/thirdparty/src/hadoop-3.2.0/*:/tmp/dist-test-taskm314YR/build/debug/bin/postgresql.jar:/tmp/dist-test-taskm314YR/thirdparty/src/ranger-2.1.0-admin/ews/webapp
I20260809 15:37:35.985884 3431 mini_ranger.cc:193] Using host: 127.3.89.212
I20260809 15:37:35.990044 3431 mini_ranger.cc:227] Ranger admin URL: http://127.3.89.212:39105
Aug 09, 2026 3:37:36 PM org.apache.ranger.server.tomcat.EmbeddedServer getKeyManagers
WARNING: Config 'ranger.keystore.file' or 'ranger.service.https.attrib.keystore.file' is not found or contains blank value
Aug 09, 2026 3:37:36 PM org.apache.ranger.server.tomcat.EmbeddedServer getTrustManagers
WARNING: Config 'ranger.truststore.file' is not found or contains blank value!
Aug 09, 2026 3:37:36 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Deriving webapp folder from catalina.base property. folder=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786289732823074-3431-0/minicluster-data/ranger-admin/ews/webapp
Aug 09, 2026 3:37:36 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Webapp file =/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786289732823074-3431-0/minicluster-data/ranger-admin/ews/webapp, webAppName = /
Aug 09, 2026 3:37:36 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Adding webapp [/] = path [/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786289732823074-3431-0/minicluster-data/ranger-admin/ews/webapp] .....
Aug 09, 2026 3:37:36 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Finished init of webapp [/] = path [/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786289732823074-3431-0/minicluster-data/ranger-admin/ews/webapp].
Aug 09, 2026 3:37:36 PM org.apache.ranger.server.tomcat.EmbeddedServer startServer
INFO: Server Name : miniranger
I20260809 15:37:37.159394 3431 mini_ranger.cc:160] Time spent starting Ranger: real 8.538s user 0.000s sys 0.012s
Aug 09, 2026 3:37:57 PM com.sun.jersey.api.core.PackagesResourceConfig init
INFO: Scanning for root resource and provider classes in the packages:
org.apache.ranger.rest
org.apache.ranger.common
xa.rest
Aug 09, 2026 3:37:57 PM com.sun.jersey.api.core.ScanningResourceConfig logClasses
INFO: Root resource classes found:
class org.apache.ranger.rest.MetricsREST
class org.apache.ranger.rest.XKeyREST
class org.apache.ranger.rest.XAuditREST
class org.apache.ranger.rest.XUserREST
class org.apache.ranger.rest.UserREST
class org.apache.ranger.rest.ServiceREST
class org.apache.ranger.rest.PublicAPIs
class org.apache.ranger.rest.TagREST
class org.apache.ranger.rest.AssetREST
class org.apache.ranger.rest.RoleREST
class org.apache.ranger.rest.SecurityZoneREST
class org.apache.ranger.rest.PublicAPIsv2
Aug 09, 2026 3:37:57 PM com.sun.jersey.api.core.ScanningResourceConfig logClasses
INFO: Provider classes found:
class org.apache.ranger.common.RangerJAXBContextResolver
Aug 09, 2026 3:37:57 PM com.sun.jersey.spi.spring.container.servlet.SpringServlet getContext
INFO: Using default applicationContext
Aug 09, 2026 3:37:57 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, assetREST, of type org.apache.ranger.rest.AssetREST as a root resource class
Aug 09, 2026 3:37:57 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, metricsREST, of type org.apache.ranger.rest.MetricsREST as a root resource class
Aug 09, 2026 3:37:57 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, publicAPIs, of type org.apache.ranger.rest.PublicAPIs as a root resource class
Aug 09, 2026 3:37:57 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, publicAPIsv2, of type org.apache.ranger.rest.PublicAPIsv2 as a root resource class
Aug 09, 2026 3:37:57 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, roleREST, of type org.apache.ranger.rest.RoleREST as a root resource class
Aug 09, 2026 3:37:57 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, securityZoneREST, of type org.apache.ranger.rest.SecurityZoneREST as a root resource class
Aug 09, 2026 3:37:57 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, serviceREST, of type org.apache.ranger.rest.ServiceREST as a root resource class
Aug 09, 2026 3:37:57 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, tagREST, of type org.apache.ranger.rest.TagREST as a root resource class
Aug 09, 2026 3:37:57 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, userREST, of type org.apache.ranger.rest.UserREST as a root resource class
Aug 09, 2026 3:37:57 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XAuditREST, of type org.apache.ranger.rest.XAuditREST as a root resource class
Aug 09, 2026 3:37:57 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XKeyREST, of type org.apache.ranger.rest.XKeyREST as a root resource class
Aug 09, 2026 3:37:57 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XUserREST, of type org.apache.ranger.rest.XUserREST as a root resource class
Aug 09, 2026 3:37:57 PM com.sun.jersey.server.impl.application.WebApplicationImpl _initiate
INFO: Initiating Jersey application, version 'Jersey: 1.19.3 10/24/2016 03:58 PM'
Aug 09, 2026 3:37:58 PM com.sun.jersey.spi.inject.Errors processErrorMessages
WARNING: The following warnings have been detected with resource and/or provider classes:
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInExcel(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse), MUST return a non-void type.
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInJson(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse,java.lang.Boolean), MUST return a non-void type.
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInCsv(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse) throws java.io.IOException, MUST return a non-void type.
I20260809 15:37:59.254391 3431 mini_ranger.cc:261] Created Kudu service
WARNING: no policy specified for rangerkms/127.3.89.212@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangerkms/127.3.89.212@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangerkms/127.3.89.212@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786289732823074-3431-0/krb5kdc/rangerkms_127.3.89.212@KRBTEST.COM.keytab.
Entry for principal rangerkms/127.3.89.212@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786289732823074-3431-0/krb5kdc/rangerkms_127.3.89.212@KRBTEST.COM.keytab.
WARNING: no policy specified for HTTP/127.3.89.212@KRBTEST.COM; defaulting to no policy
add_principal: Principal or policy already exists while creating "HTTP/127.3.89.212@KRBTEST.COM".
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal HTTP/127.3.89.212@KRBTEST.COM with kvno 3, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786289732823074-3431-0/krb5kdc/HTTP_127.3.89.212@KRBTEST.COM.keytab.
Entry for principal HTTP/127.3.89.212@KRBTEST.COM with kvno 3, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786289732823074-3431-0/krb5kdc/HTTP_127.3.89.212@KRBTEST.COM.keytab.
WARNING: no policy specified for keyadmin@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "keyadmin@KRBTEST.COM" created.
Aug 09 15:37:59 dist-test-slave-2w3w krb5kdc[16078](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786289879, etypes {rep=17 tkt=17 ses=17}, keyadmin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for keyadmin@KRBTEST.COM:
I20260809 15:37:59.360165 3431 mini_ranger_kms.cc:208] Starting Ranger KMS...
I20260809 15:37:59.371166 3431 mini_ranger_kms.cc:78] Created minirangerkms Postgres user
I20260809 15:38:00.226687 3431 mini_ranger_kms.cc:81] Created rangerkms Postgres database
I20260809 15:38:00.226786 3431 mini_ranger_kms.cc:226] Starting Ranger KMS out of /tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786289732823074-3431-0/minicluster-data/ranger-kms
I20260809 15:38:00.226806 3431 mini_ranger_kms.cc:227] Using postgres at 127.3.89.212:44657
2026-08-09 15:38:00,410 [I] DB FLAVOR :POSTGRES
2026-08-09 15:38:00,410 [I] --------- Verifying Ranger DB connection ---------
2026-08-09 15:38:00,410 [I] Checking connection
2026-08-09 15:38:00,410 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskm314YR/build/debug/bin/postgresql.jar:/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786289732823074-3431-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.3.89.212:44657/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -query "SELECT 1;"
2026-08-09 15:38:00,646 [I] connection success
2026-08-09 15:38:00,646 [I] --------- Verifying Ranger DB tables ---------
2026-08-09 15:38:00,646 [I] Verifying table ranger_masterkey in database rangerkms
2026-08-09 15:38:00,647 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskm314YR/build/debug/bin/postgresql.jar:/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786289732823074-3431-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.3.89.212:44657/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='rangerkms' and table_name = 'ranger_masterkey') as temp;"
2026-08-09 15:38:00,911 [I] Table ranger_masterkey does not exist in database rangerkms
2026-08-09 15:38:00,911 [I] --------- Importing Ranger Core DB Schema ---------
2026-08-09 15:38:00,911 [I] Importing db schema to database rangerkms from file: kms_core_db_postgres.sql
2026-08-09 15:38:00,912 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskm314YR/build/debug/bin/postgresql.jar:/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786289732823074-3431-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.3.89.212:44657/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -input /tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786289732823074-3431-0/minicluster-data/ranger-kms/db/postgres/kms_core_db_postgres.sql
2026-08-09 15:38:01,179 [I] kms_core_db_postgres.sql DB schema imported successfully
I20260809 15:38:01.680223 3431 mini_ranger_kms.cc:310] Created kms service
I20260809 15:38:01.867743 3431 mini_ranger_kms.cc:326] Created kudu user
I20260809 15:38:01.911702 3431 mini_ranger_kms.cc:343] Created rangerkms user
Aug 09 15:38:01 dist-test-slave-2w3w krb5kdc[16078](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786289879, etypes {rep=17 tkt=17 ses=17}, keyadmin@KRBTEST.COM for HTTP/127.3.89.212@KRBTEST.COM
I20260809 15:38:02.304056 3431 mini_ranger_kms.cc:384] Added ranger policy
I20260809 15:38:02.304227 3431 mini_ranger_kms.cc:238] Using RangerKMS classpath: /tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786289732823074-3431-0/minicluster-data/ranger-kms:/tmp/dist-test-taskm314YR/build/debug/bin/postgresql.jar:/tmp/dist-test-taskm314YR/thirdparty/src/ranger-2.1.0-kms/ews/webapp/WEB-INF/classes/lib/*:/tmp/dist-test-taskm314YR/thirdparty/src/ranger-2.1.0-kms/ews/webapp/lib/*:/usr/lib/jvm/java-1.8.0-openjdk-amd64/lib/*
I20260809 15:38:02.304252 3431 mini_ranger_kms.cc:240] Using host: 127.3.89.212
I20260809 15:38:02.309131 3431 mini_ranger_kms.cc:276] Ranger KMS PID: 16491
I20260809 15:38:02.309214 3431 mini_ranger_kms.cc:277] Ranger KMS URL: http://127.3.89.212:58915
Aug 09, 2026 3:38:03 PM org.apache.ranger.server.tomcat.EmbeddedServer getKeyManagers
WARNING: Config 'ranger.keystore.file' or 'ranger.service.https.attrib.keystore.file' is not found or contains blank value
Aug 09, 2026 3:38:03 PM org.apache.ranger.server.tomcat.EmbeddedServer getTrustManagers
WARNING: Config 'ranger.truststore.file' is not found or contains blank value!
Aug 09, 2026 3:38:03 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Webapp file =/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786289732823074-3431-0/minicluster-data/ranger-kms/ews/webapp, webAppName = /kms
Aug 09, 2026 3:38:03 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Adding webapp [/kms] = path [/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786289732823074-3431-0/minicluster-data/ranger-kms/ews/webapp] .....
Aug 09, 2026 3:38:03 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Finished init of webapp [/kms] = path [/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786289732823074-3431-0/minicluster-data/ranger-kms/ews/webapp].
Aug 09, 2026 3:38:03 PM org.apache.ranger.server.tomcat.EmbeddedServer startServer
INFO: Server Name : minirangerkms
Aug 09 15:38:04 dist-test-slave-2w3w krb5kdc[16078](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1786289884, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.3.89.212@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 15:38:04.683749 3431 mini_ranger_kms.cc:207] Time spent starting Ranger KMS: real 5.324s user 0.010s sys 0.159s
I20260809 15:38:04.683904 3431 mini_ranger_kms.cc:397] {"name":"kuduclusterkey","cipher":"AES/CTR/NoPadding","length":128,"description":"kuduclusterkey"}
I20260809 15:38:04.683987 3431 mini_ranger_kms.cc:401] 127.3.89.212:58915/kms/v1/keys
Aug 09 15:38:04 dist-test-slave-2w3w krb5kdc[16078](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786289884, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.3.89.212@KRBTEST.COM for HTTP/127.3.89.212@KRBTEST.COM
Aug 09 15:38:05 dist-test-slave-2w3w krb5kdc[16078](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786289884, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.3.89.212@KRBTEST.COM for HTTP/127.3.89.212@KRBTEST.COM
Aug 09 15:38:07 dist-test-slave-2w3w krb5kdc[16078](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786289887, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
WARNING: no policy specified for kudu/127.3.89.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.3.89.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.3.89.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786289732823074-3431-0/krb5kdc/kudu_127.3.89.254.keytab.
Entry for principal kudu/127.3.89.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786289732823074-3431-0/krb5kdc/kudu_127.3.89.254.keytab.
I20260809 15:38:07.600701 3431 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskm314YR/build/debug/bin/kudu
/tmp/dist-test-taskm314YR/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786289732823074-3431-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786289732823074-3431-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786289732823074-3431-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786289732823074-3431-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.3.89.254:33879
--webserver_interface=127.3.89.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786289732823074-3431-0/krb5kdc/kudu_127.3.89.254.keytab
--principal=kudu/127.3.89.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.3.89.212:35415
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.3.89.254:33879
--ranger_config_path=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786289732823074-3431-0/minicluster-data/ranger-client
--trusted_user_acl=test-admin
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.3.89.212:58915/kms
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786289732823074-3431-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786289732823074-3431-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786289732823074-3431-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 15:38:07.712363 16547 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 15:38:07.712658 16547 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 15:38:07.712739 16547 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 15:38:07.716921 16547 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 15:38:07.717020 16547 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 15:38:07.717058 16547 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 15:38:07.717116 16547 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 15:38:07.717154 16547 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 15:38:07.722563 16547 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.3.89.212:35415
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786289732823074-3431-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786289732823074-3431-0/minicluster-data/master-0/wal
--ranger_kms_url=127.3.89.212:58915/kms
--trusted_user_acl=<redacted>
--ipki_ca_key_size=768
--master_addresses=127.3.89.254:33879
--ranger_config_path=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786289732823074-3431-0/minicluster-data/ranger-client
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786289732823074-3431-0/krb5kdc/kudu_127.3.89.254.keytab
--principal=kudu/127.3.89.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.3.89.254:33879
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786289732823074-3431-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.3.89.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.16547
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786289732823074-3431-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 09 Aug 2026 14:46:55 UTC on ccycloud.quasar-yswkmf.root.comops.site
build id 4576540
I20260809 15:38:07.723628 16547 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 15:38:07.724591 16547 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 15:38:07.734475 16553 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 15:38:07.734488 16552 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 15:38:07.734935 16547 server_base.cc:1034] running on GCE node
W20260809 15:38:07.735004 16555 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 15:38:07.735592 16547 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 15:38:07.736781 16547 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 15:38:07.737965 16547 hybrid_clock.cc:648] HybridClock initialized: now 1786289887737945 us; error 35 us; skew 500 ppm
Aug 09 15:38:07 dist-test-slave-2w3w krb5kdc[16078](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786289887, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.89.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 15:38:07.740995 16547 init.cc:377] Logged in from keytab as kudu/127.3.89.254@KRBTEST.COM (short username kudu)
I20260809 15:38:07.741973 16547 webserver.cc:460] Webserver started at http://127.3.89.254:43997/ using document root <none> and password file <none>
I20260809 15:38:07.742486 16547 fs_manager.cc:359] Metadata directory not provided
I20260809 15:38:07.742569 16547 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 15:38:07.742785 16547 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 09 15:38:07 dist-test-slave-2w3w krb5kdc[16078](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786289887, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.89.254@KRBTEST.COM for HTTP/127.3.89.212@KRBTEST.COM
I20260809 15:38:07.791662 16547 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786289732823074-3431-0/minicluster-data/master-0/data/instance:
uuid: "e056d1c2456c4d4287b5ca44155ff2f2"
format_stamp: "Formatted at 2026-08-09 15:38:07 on dist-test-slave-2w3w"
server_key: "50d6a8d215c64ee018ec656030f12b05"
server_key_iv: "2d4490c715c62af1278c805334c53eab"
server_key_version: "kuduclusterkey@0"
I20260809 15:38:07.792309 16547 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786289732823074-3431-0/minicluster-data/master-0/wal/instance:
uuid: "e056d1c2456c4d4287b5ca44155ff2f2"
format_stamp: "Formatted at 2026-08-09 15:38:07 on dist-test-slave-2w3w"
server_key: "50d6a8d215c64ee018ec656030f12b05"
server_key_iv: "2d4490c715c62af1278c805334c53eab"
server_key_version: "kuduclusterkey@0"
I20260809 15:38:07.796449 16547 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.000s
I20260809 15:38:07.810524 16563 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 15:38:07.811961 16547 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260809 15:38:07.812114 16547 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786289732823074-3431-0/minicluster-data/master-0/data,/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786289732823074-3431-0/minicluster-data/master-0/wal
uuid: "e056d1c2456c4d4287b5ca44155ff2f2"
format_stamp: "Formatted at 2026-08-09 15:38:07 on dist-test-slave-2w3w"
server_key: "50d6a8d215c64ee018ec656030f12b05"
server_key_iv: "2d4490c715c62af1278c805334c53eab"
server_key_version: "kuduclusterkey@0"
I20260809 15:38:07.812270 16547 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786289732823074-3431-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786289732823074-3431-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786289732823074-3431-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 15:38:07.827571 16547 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 15:38:07.830255 16547 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 15:38:07.830454 16547 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 15:38:07.838472 16547 rpc_server.cc:307] RPC server started. Bound to: 127.3.89.254:33879
I20260809 15:38:07.838481 16625 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.3.89.254:33879 every 8 connection(s)
I20260809 15:38:07.839704 16547 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786289732823074-3431-0/minicluster-data/master-0/data/info.pb
I20260809 15:38:07.842362 16626 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 15:38:07.847553 16626 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e056d1c2456c4d4287b5ca44155ff2f2: Bootstrap starting.
I20260809 15:38:07.848991 3431 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskm314YR/build/debug/bin/kudu as pid 16547
I20260809 15:38:07.849109 3431 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786289732823074-3431-0/minicluster-data/master-0/wal/instance
I20260809 15:38:07.849416 3431 external_mini_cluster.cc:1428] Setting key 7afc82f83fec64ca32c64f4a1adb012f
I20260809 15:38:07.850219 16626 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P e056d1c2456c4d4287b5ca44155ff2f2: Neither blocks nor log segments found. Creating new log.
I20260809 15:38:07.851099 16626 log.cc:824] T 00000000000000000000000000000000 P e056d1c2456c4d4287b5ca44155ff2f2: Log is configured to *not* fsync() on all Append() calls
I20260809 15:38:07.853053 16626 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e056d1c2456c4d4287b5ca44155ff2f2: No bootstrap required, opened a new log
I20260809 15:38:07.857026 16626 raft_consensus.cc:348] T 00000000000000000000000000000000 P e056d1c2456c4d4287b5ca44155ff2f2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e056d1c2456c4d4287b5ca44155ff2f2" member_type: VOTER last_known_addr { host: "127.3.89.254" port: 33879 } }
I20260809 15:38:07.857352 16626 raft_consensus.cc:374] T 00000000000000000000000000000000 P e056d1c2456c4d4287b5ca44155ff2f2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 15:38:07.857455 16626 raft_consensus.cc:729] T 00000000000000000000000000000000 P e056d1c2456c4d4287b5ca44155ff2f2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e056d1c2456c4d4287b5ca44155ff2f2, State: Initialized, Role: FOLLOWER
I20260809 15:38:07.857970 16626 consensus_queue.cc:260] T 00000000000000000000000000000000 P e056d1c2456c4d4287b5ca44155ff2f2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e056d1c2456c4d4287b5ca44155ff2f2" member_type: VOTER last_known_addr { host: "127.3.89.254" port: 33879 } }
I20260809 15:38:07.858146 16626 raft_consensus.cc:388] T 00000000000000000000000000000000 P e056d1c2456c4d4287b5ca44155ff2f2 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 15:38:07.858249 16626 raft_consensus.cc:482] T 00000000000000000000000000000000 P e056d1c2456c4d4287b5ca44155ff2f2 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 15:38:07.858376 16626 raft_consensus.cc:3037] T 00000000000000000000000000000000 P e056d1c2456c4d4287b5ca44155ff2f2 [term 0 FOLLOWER]: Advancing to term 1
I20260809 15:38:07.859535 16626 raft_consensus.cc:504] T 00000000000000000000000000000000 P e056d1c2456c4d4287b5ca44155ff2f2 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e056d1c2456c4d4287b5ca44155ff2f2" member_type: VOTER last_known_addr { host: "127.3.89.254" port: 33879 } }
Aug 09 15:38:07 dist-test-slave-2w3w krb5kdc[16078](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786289887, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.3.89.254@KRBTEST.COM
I20260809 15:38:07.859963 16626 leader_election.cc:302] T 00000000000000000000000000000000 P e056d1c2456c4d4287b5ca44155ff2f2 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: e056d1c2456c4d4287b5ca44155ff2f2; no voters:
I20260809 15:38:07.860318 16626 leader_election.cc:288] T 00000000000000000000000000000000 P e056d1c2456c4d4287b5ca44155ff2f2 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 15:38:07.860460 16633 raft_consensus.cc:2781] T 00000000000000000000000000000000 P e056d1c2456c4d4287b5ca44155ff2f2 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 15:38:07.860733 16633 raft_consensus.cc:686] T 00000000000000000000000000000000 P e056d1c2456c4d4287b5ca44155ff2f2 [term 1 LEADER]: Becoming Leader. State: Replica: e056d1c2456c4d4287b5ca44155ff2f2, State: Running, Role: LEADER
I20260809 15:38:07.861068 16633 consensus_queue.cc:237] T 00000000000000000000000000000000 P e056d1c2456c4d4287b5ca44155ff2f2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e056d1c2456c4d4287b5ca44155ff2f2" member_type: VOTER last_known_addr { host: "127.3.89.254" port: 33879 } }
I20260809 15:38:07.861784 16626 sys_catalog.cc:564] T 00000000000000000000000000000000 P e056d1c2456c4d4287b5ca44155ff2f2 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 15:38:07.862291 16633 sys_catalog.cc:455] T 00000000000000000000000000000000 P e056d1c2456c4d4287b5ca44155ff2f2 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "e056d1c2456c4d4287b5ca44155ff2f2" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e056d1c2456c4d4287b5ca44155ff2f2" member_type: VOTER last_known_addr { host: "127.3.89.254" port: 33879 } } }
I20260809 15:38:07.862524 16633 sys_catalog.cc:458] T 00000000000000000000000000000000 P e056d1c2456c4d4287b5ca44155ff2f2 [sys.catalog]: This master's current role is: LEADER
I20260809 15:38:07.862778 16633 sys_catalog.cc:455] T 00000000000000000000000000000000 P e056d1c2456c4d4287b5ca44155ff2f2 [sys.catalog]: SysCatalogTable state changed. Reason: New leader e056d1c2456c4d4287b5ca44155ff2f2. Latest consensus state: current_term: 1 leader_uuid: "e056d1c2456c4d4287b5ca44155ff2f2" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e056d1c2456c4d4287b5ca44155ff2f2" member_type: VOTER last_known_addr { host: "127.3.89.254" port: 33879 } } }
I20260809 15:38:07.862910 16633 sys_catalog.cc:458] T 00000000000000000000000000000000 P e056d1c2456c4d4287b5ca44155ff2f2 [sys.catalog]: This master's current role is: LEADER
I20260809 15:38:07.863230 16641 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 15:38:07.866402 16641 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 15:38:07.867173 16626 ranger_client.cc:318] Using new properties file: /tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786289732823074-3431-0/minicluster-data/master-0/logs/kudu-ranger-subprocess-log4j2.properties
I20260809 15:38:07.866444 16631 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:38:07.850580 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55942 (local address 127.3.89.254:33879)
0809 15:38:07.851067 (+ 487us) server_negotiation.cc:206] Beginning negotiation
0809 15:38:07.851086 (+ 19us) server_negotiation.cc:399] Waiting for connection header
0809 15:38:07.851128 (+ 42us) server_negotiation.cc:407] Connection header received
0809 15:38:07.851900 (+ 772us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 15:38:07.851922 (+ 22us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 15:38:07.852304 (+ 382us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 15:38:07.852671 (+ 367us) server_negotiation.cc:226] Negotiated authn=SASL
0809 15:38:07.853615 (+ 944us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:38:07.855112 (+ 1497us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:38:07.856373 (+ 1261us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:38:07.857060 (+ 687us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:38:07.857214 (+ 154us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:38:07.860489 (+ 3275us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 15:38:07.860495 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 15:38:07.860511 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 15:38:07.860540 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0809 15:38:07.862553 (+ 2013us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 15:38:07.863130 (+ 577us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 15:38:07.863135 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 15:38:07.863141 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0809 15:38:07.863274 (+ 133us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 15:38:07.863637 (+ 363us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 15:38:07.863640 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 15:38:07.863642 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 15:38:07.864191 (+ 549us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 15:38:07.865094 (+ 903us) server_negotiation.cc:1035] Waiting for connection context
0809 15:38:07.865238 (+ 144us) server_negotiation.cc:299] Negotiation successful
0809 15:38:07.865522 (+ 284us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":317,"thread_start_us":135,"threads_started":1}
I20260809 15:38:07.877701 16641 catalog_manager.cc:1347] Generated new cluster ID: 160fa241659546dc8e93ecac319a6270
I20260809 15:38:07.877789 16641 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 15:38:07.901465 16641 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 15:38:07.902935 16641 catalog_manager.cc:1504] Loading token signing keys...
I20260809 15:38:07.908982 16641 catalog_manager.cc:5868] T 00000000000000000000000000000000 P e056d1c2456c4d4287b5ca44155ff2f2: Generated new TSK 0
I20260809 15:38:07.909808 16641 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.3.89.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.3.89.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.3.89.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786289732823074-3431-0/krb5kdc/kudu_127.3.89.193.keytab.
Entry for principal kudu/127.3.89.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786289732823074-3431-0/krb5kdc/kudu_127.3.89.193.keytab.
I20260809 15:38:08.643129 3431 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskm314YR/build/debug/bin/kudu
/tmp/dist-test-taskm314YR/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786289732823074-3431-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786289732823074-3431-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786289732823074-3431-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786289732823074-3431-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.3.89.212:58915/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.3.89.193:0
--local_ip_for_outbound_sockets=127.3.89.193
--webserver_interface=127.3.89.193
--webserver_port=0
--tserver_master_addrs=127.3.89.254:33879
--keytab_file=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786289732823074-3431-0/krb5kdc/kudu_127.3.89.193.keytab
--principal=kudu/127.3.89.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.3.89.212:35415
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786289732823074-3431-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786289732823074-3431-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786289732823074-3431-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 15:38:08.796029 16675 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 15:38:08.796491 16675 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 15:38:08.796592 16675 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 15:38:08.802873 16675 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 15:38:08.802978 16675 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 15:38:08.803076 16675 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.3.89.193
I20260809 15:38:08.811041 16675 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.3.89.212:35415
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786289732823074-3431-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786289732823074-3431-0/minicluster-data/ts-0/wal
--ranger_kms_url=127.3.89.212:58915/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786289732823074-3431-0/krb5kdc/kudu_127.3.89.193.keytab
--principal=kudu/127.3.89.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.3.89.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786289732823074-3431-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.3.89.193
--webserver_port=0
--tserver_master_addrs=127.3.89.254:33879
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.16675
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.3.89.193
--log_dir=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786289732823074-3431-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 09 Aug 2026 14:46:55 UTC on ccycloud.quasar-yswkmf.root.comops.site
build id 4576540
I20260809 15:38:08.812409 16675 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 15:38:08.813776 16675 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 15:38:08.826022 16686 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 15:38:08.826030 16684 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 15:38:08.826041 16682 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 15:38:08.826336 16675 server_base.cc:1034] running on GCE node
I20260809 15:38:08.826905 16675 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 15:38:08.827571 16675 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 15:38:08.828759 16675 hybrid_clock.cc:648] HybridClock initialized: now 1786289888828737 us; error 42 us; skew 500 ppm
Aug 09 15:38:08 dist-test-slave-2w3w krb5kdc[16078](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786289888, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.89.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 15:38:08.832283 16675 init.cc:377] Logged in from keytab as kudu/127.3.89.193@KRBTEST.COM (short username kudu)
I20260809 15:38:08.833074 16675 webserver.cc:460] Webserver started at http://127.3.89.193:34171/ using document root <none> and password file <none>
I20260809 15:38:08.833595 16675 fs_manager.cc:359] Metadata directory not provided
I20260809 15:38:08.833673 16675 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 15:38:08.833882 16675 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 09 15:38:08 dist-test-slave-2w3w krb5kdc[16078](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786289888, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.89.193@KRBTEST.COM for HTTP/127.3.89.212@KRBTEST.COM
I20260809 15:38:08.858187 16675 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786289732823074-3431-0/minicluster-data/ts-0/data/instance:
uuid: "28b33706caad4d379f73b37cbf0656c8"
format_stamp: "Formatted at 2026-08-09 15:38:08 on dist-test-slave-2w3w"
server_key: "20b284a3ff0e806bc9af0c499a84de90"
server_key_iv: "6cacfa3671ac7d86abcbe5c4ece48628"
server_key_version: "kuduclusterkey@0"
I20260809 15:38:08.858917 16675 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786289732823074-3431-0/minicluster-data/ts-0/wal/instance:
uuid: "28b33706caad4d379f73b37cbf0656c8"
format_stamp: "Formatted at 2026-08-09 15:38:08 on dist-test-slave-2w3w"
server_key: "20b284a3ff0e806bc9af0c499a84de90"
server_key_iv: "6cacfa3671ac7d86abcbe5c4ece48628"
server_key_version: "kuduclusterkey@0"
I20260809 15:38:08.866784 16675 fs_manager.cc:691] Time spent creating directory manager: real 0.008s user 0.003s sys 0.000s
I20260809 15:38:08.883735 16694 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 15:38:08.885381 16675 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.003s sys 0.000s
I20260809 15:38:08.885530 16675 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786289732823074-3431-0/minicluster-data/ts-0/data,/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786289732823074-3431-0/minicluster-data/ts-0/wal
uuid: "28b33706caad4d379f73b37cbf0656c8"
format_stamp: "Formatted at 2026-08-09 15:38:08 on dist-test-slave-2w3w"
server_key: "20b284a3ff0e806bc9af0c499a84de90"
server_key_iv: "6cacfa3671ac7d86abcbe5c4ece48628"
server_key_version: "kuduclusterkey@0"
I20260809 15:38:08.885756 16675 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786289732823074-3431-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786289732823074-3431-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786289732823074-3431-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 15:38:08.921664 16675 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 15:38:08.925387 16675 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 15:38:08.925625 16675 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 15:38:08.926376 16675 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 15:38:08.927524 16675 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 15:38:08.927641 16675 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 15:38:08.927770 16675 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 15:38:08.927876 16675 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 15:38:08.942299 16675 rpc_server.cc:307] RPC server started. Bound to: 127.3.89.193:37409
I20260809 15:38:08.943665 16675 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786289732823074-3431-0/minicluster-data/ts-0/data/info.pb
I20260809 15:38:08.944669 16817 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.3.89.193:37409 every 8 connection(s)
I20260809 15:38:08.949172 3431 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskm314YR/build/debug/bin/kudu as pid 16675
I20260809 15:38:08.949288 3431 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786289732823074-3431-0/minicluster-data/ts-0/wal/instance
I20260809 15:38:08.949576 3431 external_mini_cluster.cc:1428] Setting key 0a98ae89d524aa41e3852663b0aef4ba
Aug 09 15:38:08 dist-test-slave-2w3w krb5kdc[16078](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786289888, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.89.193@KRBTEST.COM for kudu/127.3.89.254@KRBTEST.COM
I20260809 15:38:08.962376 16821 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:38:08.947216 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.89.254:33879 (local address 127.3.89.193:55693)
0809 15:38:08.947768 (+ 552us) negotiation.cc:107] Waiting for socket to connect
0809 15:38:08.947829 (+ 61us) client_negotiation.cc:174] Beginning negotiation
0809 15:38:08.948872 (+ 1043us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 15:38:08.949656 (+ 784us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 15:38:08.949665 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 15:38:08.950039 (+ 374us) client_negotiation.cc:189] Negotiated authn=SASL
0809 15:38:08.950694 (+ 655us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:38:08.950714 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:38:08.951926 (+ 1212us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:38:08.951930 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:38:08.953338 (+ 1408us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:38:08.953349 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:38:08.954143 (+ 794us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:38:08.954147 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:38:08.954341 (+ 194us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:38:08.955109 (+ 768us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 15:38:08.955130 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0809 15:38:08.956920 (+ 1790us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 15:38:08.959212 (+ 2292us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 15:38:08.959217 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 15:38:08.959233 (+ 16us) client_negotiation.cc:760] calling sasl_client_step()
0809 15:38:08.959571 (+ 338us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 15:38:08.959897 (+ 326us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 15:38:08.959900 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 15:38:08.959902 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 15:38:08.960075 (+ 173us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 15:38:08.960536 (+ 461us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 15:38:08.960542 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 15:38:08.960820 (+ 278us) client_negotiation.cc:769] Sending connection context
0809 15:38:08.961024 (+ 204us) client_negotiation.cc:240] Negotiation successful
0809 15:38:08.961354 (+ 330us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":315,"thread_start_us":137,"threads_started":1}
I20260809 15:38:08.963438 16820 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:38:08.946681 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.89.193:55693 (local address 127.3.89.254:33879)
0809 15:38:08.947472 (+ 791us) server_negotiation.cc:206] Beginning negotiation
0809 15:38:08.947477 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 15:38:08.949171 (+ 1694us) server_negotiation.cc:407] Connection header received
0809 15:38:08.949224 (+ 53us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 15:38:08.949227 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 15:38:08.949281 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 15:38:08.949364 (+ 83us) server_negotiation.cc:226] Negotiated authn=SASL
0809 15:38:08.950872 (+ 1508us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:38:08.951768 (+ 896us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:38:08.953487 (+ 1719us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:38:08.953850 (+ 363us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:38:08.953929 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:38:08.957131 (+ 3202us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 15:38:08.957135 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 15:38:08.957137 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 15:38:08.957163 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0809 15:38:08.959080 (+ 1917us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 15:38:08.959691 (+ 611us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 15:38:08.959695 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 15:38:08.959697 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 15:38:08.959800 (+ 103us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 15:38:08.960180 (+ 380us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 15:38:08.960183 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 15:38:08.960185 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 15:38:08.960390 (+ 205us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 15:38:08.960482 (+ 92us) server_negotiation.cc:1035] Waiting for connection context
0809 15:38:08.963022 (+ 2540us) server_negotiation.cc:299] Negotiation successful
0809 15:38:08.963178 (+ 156us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":699,"thread_start_us":87,"threads_started":1}
I20260809 15:38:08.964895 16818 heartbeater.cc:347] Connected to a master server at 127.3.89.254:33879
I20260809 15:38:08.965387 16818 heartbeater.cc:464] Registering TS with master...
I20260809 15:38:08.966054 16818 heartbeater.cc:511] Master 127.3.89.254:33879 requested a full tablet report, sending...
WARNING: no policy specified for kudu/127.3.89.194@KRBTEST.COM; defaulting to no policy
I20260809 15:38:08.967963 16581 ts_manager.cc:194] Registered new tserver with Master: 28b33706caad4d379f73b37cbf0656c8 (127.3.89.193:37409)
I20260809 15:38:08.969465 16581 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.3.89.193@KRBTEST.COM'} at 127.3.89.193:55693
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.3.89.194@KRBTEST.COM" created.
Aug 09 15:38:08 dist-test-slave-2w3w krb5kdc[16078](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1786289888, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.89.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.3.89.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786289732823074-3431-0/krb5kdc/kudu_127.3.89.194.keytab.
Entry for principal kudu/127.3.89.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786289732823074-3431-0/krb5kdc/kudu_127.3.89.194.keytab.
I20260809 15:38:08.991364 3431 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskm314YR/build/debug/bin/kudu
/tmp/dist-test-taskm314YR/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786289732823074-3431-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786289732823074-3431-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786289732823074-3431-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786289732823074-3431-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.3.89.212:58915/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.3.89.194:0
--local_ip_for_outbound_sockets=127.3.89.194
--webserver_interface=127.3.89.194
--webserver_port=0
--tserver_master_addrs=127.3.89.254:33879
--keytab_file=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786289732823074-3431-0/krb5kdc/kudu_127.3.89.194.keytab
--principal=kudu/127.3.89.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.3.89.212:35415
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786289732823074-3431-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786289732823074-3431-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786289732823074-3431-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 15:38:09.129989 16824 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 15:38:09.130271 16824 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 15:38:09.130347 16824 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 15:38:09.134784 16824 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 15:38:09.134871 16824 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 15:38:09.134992 16824 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.3.89.194
I20260809 15:38:09.140808 16824 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.3.89.212:35415
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786289732823074-3431-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786289732823074-3431-0/minicluster-data/ts-1/wal
--ranger_kms_url=127.3.89.212:58915/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786289732823074-3431-0/krb5kdc/kudu_127.3.89.194.keytab
--principal=kudu/127.3.89.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.3.89.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786289732823074-3431-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.3.89.194
--webserver_port=0
--tserver_master_addrs=127.3.89.254:33879
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.16824
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.3.89.194
--log_dir=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786289732823074-3431-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 09 Aug 2026 14:46:55 UTC on ccycloud.quasar-yswkmf.root.comops.site
build id 4576540
I20260809 15:38:09.141891 16824 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 15:38:09.142881 16824 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 15:38:09.153719 16834 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 15:38:09.154008 16830 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 15:38:09.153729 16832 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 15:38:09.155424 16824 server_base.cc:1034] running on GCE node
I20260809 15:38:09.155978 16824 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 15:38:09.156708 16824 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 15:38:09.157912 16824 hybrid_clock.cc:648] HybridClock initialized: now 1786289889157885 us; error 43 us; skew 500 ppm
Aug 09 15:38:09 dist-test-slave-2w3w krb5kdc[16078](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786289889, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.89.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 15:38:09.160801 16824 init.cc:377] Logged in from keytab as kudu/127.3.89.194@KRBTEST.COM (short username kudu)
I20260809 15:38:09.161888 16824 webserver.cc:460] Webserver started at http://127.3.89.194:39909/ using document root <none> and password file <none>
I20260809 15:38:09.162590 16824 fs_manager.cc:359] Metadata directory not provided
I20260809 15:38:09.162688 16824 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 15:38:09.163013 16824 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 09 15:38:09 dist-test-slave-2w3w krb5kdc[16078](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786289889, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.89.194@KRBTEST.COM for HTTP/127.3.89.212@KRBTEST.COM
I20260809 15:38:09.185335 16824 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786289732823074-3431-0/minicluster-data/ts-1/data/instance:
uuid: "e7f918425ade4bcbb5ff5f6ece608c08"
format_stamp: "Formatted at 2026-08-09 15:38:09 on dist-test-slave-2w3w"
server_key: "7b630996cc7dc8590df4d797011a3859"
server_key_iv: "b7e25b0825ec43744a7dddf88350d69d"
server_key_version: "kuduclusterkey@0"
I20260809 15:38:09.186102 16824 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786289732823074-3431-0/minicluster-data/ts-1/wal/instance:
uuid: "e7f918425ade4bcbb5ff5f6ece608c08"
format_stamp: "Formatted at 2026-08-09 15:38:09 on dist-test-slave-2w3w"
server_key: "7b630996cc7dc8590df4d797011a3859"
server_key_iv: "b7e25b0825ec43744a7dddf88350d69d"
server_key_version: "kuduclusterkey@0"
I20260809 15:38:09.192304 16824 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.006s sys 0.000s
I20260809 15:38:09.215411 16841 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 15:38:09.217356 16824 fs_manager.cc:725] Time spent opening block manager: real 0.010s user 0.002s sys 0.000s
I20260809 15:38:09.217559 16824 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786289732823074-3431-0/minicluster-data/ts-1/data,/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786289732823074-3431-0/minicluster-data/ts-1/wal
uuid: "e7f918425ade4bcbb5ff5f6ece608c08"
format_stamp: "Formatted at 2026-08-09 15:38:09 on dist-test-slave-2w3w"
server_key: "7b630996cc7dc8590df4d797011a3859"
server_key_iv: "b7e25b0825ec43744a7dddf88350d69d"
server_key_version: "kuduclusterkey@0"
I20260809 15:38:09.217799 16824 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786289732823074-3431-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786289732823074-3431-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786289732823074-3431-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 15:38:09.239481 16824 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 15:38:09.243196 16824 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 15:38:09.243448 16824 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 15:38:09.244130 16824 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 15:38:09.245218 16824 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 15:38:09.245340 16824 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 15:38:09.245443 16824 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 15:38:09.245508 16824 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 15:38:09.259605 16824 rpc_server.cc:307] RPC server started. Bound to: 127.3.89.194:39621
I20260809 15:38:09.259915 16961 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.3.89.194:39621 every 8 connection(s)
I20260809 15:38:09.260792 16824 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786289732823074-3431-0/minicluster-data/ts-1/data/info.pb
I20260809 15:38:09.271023 3431 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskm314YR/build/debug/bin/kudu as pid 16824
I20260809 15:38:09.271153 3431 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786289732823074-3431-0/minicluster-data/ts-1/wal/instance
I20260809 15:38:09.271462 3431 external_mini_cluster.cc:1428] Setting key 514923bce657e27327defdbd2b301273
Aug 09 15:38:09 dist-test-slave-2w3w krb5kdc[16078](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786289889, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.89.194@KRBTEST.COM for kudu/127.3.89.254@KRBTEST.COM
I20260809 15:38:09.281045 16964 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:38:09.263544 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.89.254:33879 (local address 127.3.89.194:59871)
0809 15:38:09.264081 (+ 537us) negotiation.cc:107] Waiting for socket to connect
0809 15:38:09.264140 (+ 59us) client_negotiation.cc:174] Beginning negotiation
0809 15:38:09.265185 (+ 1045us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 15:38:09.265800 (+ 615us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 15:38:09.265808 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 15:38:09.266142 (+ 334us) client_negotiation.cc:189] Negotiated authn=SASL
0809 15:38:09.266766 (+ 624us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:38:09.266784 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:38:09.268102 (+ 1318us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:38:09.268105 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:38:09.269420 (+ 1315us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:38:09.269431 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:38:09.270142 (+ 711us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:38:09.270146 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:38:09.270333 (+ 187us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:38:09.271975 (+ 1642us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 15:38:09.271996 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0809 15:38:09.275408 (+ 3412us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 15:38:09.277636 (+ 2228us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 15:38:09.277642 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 15:38:09.277669 (+ 27us) client_negotiation.cc:760] calling sasl_client_step()
0809 15:38:09.278051 (+ 382us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 15:38:09.278397 (+ 346us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 15:38:09.278400 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 15:38:09.278402 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 15:38:09.278579 (+ 177us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 15:38:09.279118 (+ 539us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 15:38:09.279125 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 15:38:09.279444 (+ 319us) client_negotiation.cc:769] Sending connection context
0809 15:38:09.279654 (+ 210us) client_negotiation.cc:240] Negotiation successful
0809 15:38:09.280000 (+ 346us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":319,"thread_start_us":133,"threads_started":1}
I20260809 15:38:09.281766 16820 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:38:09.263137 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.89.194:59871 (local address 127.3.89.254:33879)
0809 15:38:09.263278 (+ 141us) server_negotiation.cc:206] Beginning negotiation
0809 15:38:09.263283 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 15:38:09.264245 (+ 962us) server_negotiation.cc:407] Connection header received
0809 15:38:09.265436 (+ 1191us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 15:38:09.265440 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 15:38:09.265510 (+ 70us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 15:38:09.265605 (+ 95us) server_negotiation.cc:226] Negotiated authn=SASL
0809 15:38:09.266948 (+ 1343us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:38:09.267948 (+ 1000us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:38:09.269578 (+ 1630us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:38:09.269958 (+ 380us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:38:09.270067 (+ 109us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:38:09.275608 (+ 5541us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 15:38:09.275612 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 15:38:09.275615 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 15:38:09.275643 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0809 15:38:09.277501 (+ 1858us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 15:38:09.278175 (+ 674us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 15:38:09.278178 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 15:38:09.278181 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 15:38:09.278276 (+ 95us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 15:38:09.278695 (+ 419us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 15:38:09.278698 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 15:38:09.278700 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 15:38:09.278933 (+ 233us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 15:38:09.279064 (+ 131us) server_negotiation.cc:1035] Waiting for connection context
0809 15:38:09.281479 (+ 2415us) server_negotiation.cc:299] Negotiation successful
0809 15:38:09.281606 (+ 127us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":40}
I20260809 15:38:09.282898 16962 heartbeater.cc:347] Connected to a master server at 127.3.89.254:33879
I20260809 15:38:09.283175 16962 heartbeater.cc:464] Registering TS with master...
I20260809 15:38:09.283784 16962 heartbeater.cc:511] Master 127.3.89.254:33879 requested a full tablet report, sending...
I20260809 15:38:09.285127 16581 ts_manager.cc:194] Registered new tserver with Master: e7f918425ade4bcbb5ff5f6ece608c08 (127.3.89.194:39621)
I20260809 15:38:09.285975 16581 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.3.89.194@KRBTEST.COM'} at 127.3.89.194:59871
WARNING: no policy specified for kudu/127.3.89.195@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.3.89.195@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.3.89.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786289732823074-3431-0/krb5kdc/kudu_127.3.89.195.keytab.
Entry for principal kudu/127.3.89.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786289732823074-3431-0/krb5kdc/kudu_127.3.89.195.keytab.
I20260809 15:38:09.340368 3431 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskm314YR/build/debug/bin/kudu
/tmp/dist-test-taskm314YR/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786289732823074-3431-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786289732823074-3431-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786289732823074-3431-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786289732823074-3431-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.3.89.212:58915/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.3.89.195:0
--local_ip_for_outbound_sockets=127.3.89.195
--webserver_interface=127.3.89.195
--webserver_port=0
--tserver_master_addrs=127.3.89.254:33879
--keytab_file=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786289732823074-3431-0/krb5kdc/kudu_127.3.89.195.keytab
--principal=kudu/127.3.89.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.3.89.212:35415
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786289732823074-3431-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786289732823074-3431-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786289732823074-3431-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 09 15:38:09 dist-test-slave-2w3w krb5kdc[16078](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786289888, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.89.254@KRBTEST.COM for HTTP/127.3.89.212@KRBTEST.COM
W20260809 15:38:09.511348 16967 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 15:38:09.511763 16967 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 15:38:09.511862 16967 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 15:38:09.518148 16967 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 15:38:09.518265 16967 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 15:38:09.518375 16967 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.3.89.195
I20260809 15:38:09.525676 16967 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.3.89.212:35415
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786289732823074-3431-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786289732823074-3431-0/minicluster-data/ts-2/wal
--ranger_kms_url=127.3.89.212:58915/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786289732823074-3431-0/krb5kdc/kudu_127.3.89.195.keytab
--principal=kudu/127.3.89.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.3.89.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786289732823074-3431-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.3.89.195
--webserver_port=0
--tserver_master_addrs=127.3.89.254:33879
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.16967
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.3.89.195
--log_dir=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786289732823074-3431-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 09 Aug 2026 14:46:55 UTC on ccycloud.quasar-yswkmf.root.comops.site
build id 4576540
I20260809 15:38:09.527114 16967 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 15:38:09.528420 16967 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 15:38:09.539598 16972 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 15:38:09.539605 16975 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 15:38:09.539655 16973 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 15:38:09.540580 16967 server_base.cc:1034] running on GCE node
I20260809 15:38:09.540975 16967 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 15:38:09.541661 16967 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 15:38:09.542856 16967 hybrid_clock.cc:648] HybridClock initialized: now 1786289889542831 us; error 41 us; skew 500 ppm
Aug 09 15:38:09 dist-test-slave-2w3w krb5kdc[16078](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786289889, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.89.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 15:38:09.545615 16967 init.cc:377] Logged in from keytab as kudu/127.3.89.195@KRBTEST.COM (short username kudu)
I20260809 15:38:09.546557 16967 webserver.cc:460] Webserver started at http://127.3.89.195:36985/ using document root <none> and password file <none>
I20260809 15:38:09.547165 16967 fs_manager.cc:359] Metadata directory not provided
I20260809 15:38:09.547240 16967 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 15:38:09.547506 16967 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 09 15:38:09 dist-test-slave-2w3w krb5kdc[16078](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786289889, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.89.195@KRBTEST.COM for HTTP/127.3.89.212@KRBTEST.COM
I20260809 15:38:09.563593 16967 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786289732823074-3431-0/minicluster-data/ts-2/data/instance:
uuid: "216a25fe11b84aed82d3183ea221a328"
format_stamp: "Formatted at 2026-08-09 15:38:09 on dist-test-slave-2w3w"
server_key: "9ec81d0949be86d130f1070f272a8623"
server_key_iv: "317f12e9197a3b430bd33fedd0632ba2"
server_key_version: "kuduclusterkey@0"
I20260809 15:38:09.564388 16967 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786289732823074-3431-0/minicluster-data/ts-2/wal/instance:
uuid: "216a25fe11b84aed82d3183ea221a328"
format_stamp: "Formatted at 2026-08-09 15:38:09 on dist-test-slave-2w3w"
server_key: "9ec81d0949be86d130f1070f272a8623"
server_key_iv: "317f12e9197a3b430bd33fedd0632ba2"
server_key_version: "kuduclusterkey@0"
I20260809 15:38:09.574484 16967 fs_manager.cc:691] Time spent creating directory manager: real 0.010s user 0.004s sys 0.000s
I20260809 15:38:09.595368 16983 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 15:38:09.597164 16967 fs_manager.cc:725] Time spent opening block manager: real 0.007s user 0.003s sys 0.001s
I20260809 15:38:09.597306 16967 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786289732823074-3431-0/minicluster-data/ts-2/data,/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786289732823074-3431-0/minicluster-data/ts-2/wal
uuid: "216a25fe11b84aed82d3183ea221a328"
format_stamp: "Formatted at 2026-08-09 15:38:09 on dist-test-slave-2w3w"
server_key: "9ec81d0949be86d130f1070f272a8623"
server_key_iv: "317f12e9197a3b430bd33fedd0632ba2"
server_key_version: "kuduclusterkey@0"
I20260809 15:38:09.597440 16967 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786289732823074-3431-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786289732823074-3431-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786289732823074-3431-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 15:38:09.624061 16967 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 15:38:09.627687 16967 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 15:38:09.627952 16967 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 15:38:09.628661 16967 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 15:38:09.629776 16967 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 15:38:09.629882 16967 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 15:38:09.629993 16967 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 15:38:09.630060 16967 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 15:38:09.641952 16967 rpc_server.cc:307] RPC server started. Bound to: 127.3.89.195:42979
I20260809 15:38:09.642011 17101 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.3.89.195:42979 every 8 connection(s)
I20260809 15:38:09.643126 16967 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786289732823074-3431-0/minicluster-data/ts-2/data/info.pb
I20260809 15:38:09.652096 3431 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskm314YR/build/debug/bin/kudu as pid 16967
I20260809 15:38:09.652208 3431 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786289732823074-3431-0/minicluster-data/ts-2/wal/instance
I20260809 15:38:09.652485 3431 external_mini_cluster.cc:1428] Setting key b4e237236394acfb1adb2d250d00ac09
Aug 09 15:38:09 dist-test-slave-2w3w krb5kdc[16078](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786289889, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.89.195@KRBTEST.COM for kudu/127.3.89.254@KRBTEST.COM
I20260809 15:38:09.659636 16820 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:38:09.646708 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.89.195:37027 (local address 127.3.89.254:33879)
0809 15:38:09.646837 (+ 129us) server_negotiation.cc:206] Beginning negotiation
0809 15:38:09.646841 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 15:38:09.646857 (+ 16us) server_negotiation.cc:407] Connection header received
0809 15:38:09.647541 (+ 684us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 15:38:09.647545 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 15:38:09.647599 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 15:38:09.647691 (+ 92us) server_negotiation.cc:226] Negotiated authn=SASL
0809 15:38:09.649050 (+ 1359us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:38:09.649918 (+ 868us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:38:09.651574 (+ 1656us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:38:09.651932 (+ 358us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:38:09.652044 (+ 112us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:38:09.655363 (+ 3319us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 15:38:09.655367 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 15:38:09.655370 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 15:38:09.655398 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0809 15:38:09.657099 (+ 1701us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 15:38:09.657761 (+ 662us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 15:38:09.657765 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 15:38:09.657767 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 15:38:09.657858 (+ 91us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 15:38:09.658305 (+ 447us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 15:38:09.658309 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 15:38:09.658311 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 15:38:09.658541 (+ 230us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 15:38:09.658645 (+ 104us) server_negotiation.cc:1035] Waiting for connection context
0809 15:38:09.659327 (+ 682us) server_negotiation.cc:299] Negotiation successful
0809 15:38:09.659450 (+ 123us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":61}
I20260809 15:38:09.661448 17104 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:38:09.645621 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.89.254:33879 (local address 127.3.89.195:37027)
0809 15:38:09.646214 (+ 593us) negotiation.cc:107] Waiting for socket to connect
0809 15:38:09.646272 (+ 58us) client_negotiation.cc:174] Beginning negotiation
0809 15:38:09.647295 (+ 1023us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 15:38:09.647902 (+ 607us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 15:38:09.647910 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 15:38:09.648241 (+ 331us) client_negotiation.cc:189] Negotiated authn=SASL
0809 15:38:09.648867 (+ 626us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:38:09.648885 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:38:09.650069 (+ 1184us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:38:09.650073 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:38:09.651441 (+ 1368us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:38:09.651452 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:38:09.652214 (+ 762us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:38:09.652218 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:38:09.652426 (+ 208us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:38:09.653388 (+ 962us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 15:38:09.653408 (+ 20us) client_negotiation.cc:656] calling sasl_client_start()
0809 15:38:09.655179 (+ 1771us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 15:38:09.657236 (+ 2057us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 15:38:09.657242 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 15:38:09.657256 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0809 15:38:09.657611 (+ 355us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 15:38:09.657978 (+ 367us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 15:38:09.657981 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 15:38:09.657983 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 15:38:09.658171 (+ 188us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 15:38:09.658679 (+ 508us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 15:38:09.658686 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 15:38:09.659033 (+ 347us) client_negotiation.cc:769] Sending connection context
0809 15:38:09.660061 (+ 1028us) client_negotiation.cc:240] Negotiation successful
0809 15:38:09.660383 (+ 322us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":358,"thread_start_us":162,"threads_started":1}
I20260809 15:38:09.663425 17102 heartbeater.cc:347] Connected to a master server at 127.3.89.254:33879
I20260809 15:38:09.663720 17102 heartbeater.cc:464] Registering TS with master...
I20260809 15:38:09.664222 17102 heartbeater.cc:511] Master 127.3.89.254:33879 requested a full tablet report, sending...
I20260809 15:38:09.665279 16581 ts_manager.cc:194] Registered new tserver with Master: 216a25fe11b84aed82d3183ea221a328 (127.3.89.195:42979)
I20260809 15:38:09.666008 16581 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.3.89.195@KRBTEST.COM'} at 127.3.89.195:37027
I20260809 15:38:09.666725 3431 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260809 15:38:09.667635 3431 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskm314YR/build/debug/bin/kudu with pid 16675
I20260809 15:38:09.680279 3431 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskm314YR/build/debug/bin/kudu with pid 16824
I20260809 15:38:09.687379 3431 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskm314YR/build/debug/bin/kudu with pid 16967
I20260809 15:38:09.695964 3431 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskm314YR/build/debug/bin/kudu with pid 16547
I20260809 15:38:09.703214 3431 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskm314YR/build/debug/bin/kudu
/tmp/dist-test-taskm314YR/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786289732823074-3431-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786289732823074-3431-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786289732823074-3431-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786289732823074-3431-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.3.89.254:33879
--webserver_interface=127.3.89.254
--webserver_port=43997
--keytab_file=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786289732823074-3431-0/krb5kdc/kudu_127.3.89.254.keytab
--principal=kudu/127.3.89.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.3.89.212:35415
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.3.89.254:33879
--ranger_config_path=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786289732823074-3431-0/minicluster-data/ranger-client
--trusted_user_acl=test-admin
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.3.89.212:58915/kms
--rpc_trace_negotiation
--ranger_kms_url=invalid.host:1234/kms,127.3.89.212:58915/kms with env {KRB5CCNAME=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786289732823074-3431-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786289732823074-3431-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786289732823074-3431-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 15:38:09.814118 17105 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 15:38:09.814357 17105 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 15:38:09.814397 17105 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 15:38:09.818393 17105 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 15:38:09.818449 17105 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 15:38:09.818468 17105 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 15:38:09.818485 17105 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 15:38:09.818499 17105 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 15:38:09.824836 17105 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.3.89.212:35415
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786289732823074-3431-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786289732823074-3431-0/minicluster-data/master-0/wal
--ranger_kms_url=invalid.host:1234/kms,127.3.89.212:58915/kms
--trusted_user_acl=<redacted>
--ipki_ca_key_size=768
--master_addresses=127.3.89.254:33879
--ranger_config_path=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786289732823074-3431-0/minicluster-data/ranger-client
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786289732823074-3431-0/krb5kdc/kudu_127.3.89.254.keytab
--principal=kudu/127.3.89.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.3.89.254:33879
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786289732823074-3431-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.3.89.254
--webserver_port=43997
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.17105
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786289732823074-3431-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 09 Aug 2026 14:46:55 UTC on ccycloud.quasar-yswkmf.root.comops.site
build id 4576540
I20260809 15:38:09.825699 17105 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 15:38:09.826622 17105 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 15:38:09.834894 17111 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 15:38:09.834900 17113 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 15:38:09.834934 17105 server_base.cc:1034] running on GCE node
W20260809 15:38:09.834910 17110 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 15:38:09.835605 17105 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 15:38:09.836572 17105 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 15:38:09.837779 17105 hybrid_clock.cc:648] HybridClock initialized: now 1786289889837757 us; error 41 us; skew 500 ppm
Aug 09 15:38:09 dist-test-slave-2w3w krb5kdc[16078](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786289889, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.89.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 15:38:09.840502 17105 init.cc:377] Logged in from keytab as kudu/127.3.89.254@KRBTEST.COM (short username kudu)
I20260809 15:38:09.841355 17105 webserver.cc:460] Webserver started at http://127.3.89.254:43997/ using document root <none> and password file <none>
I20260809 15:38:09.841861 17105 fs_manager.cc:359] Metadata directory not provided
I20260809 15:38:09.841954 17105 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
Aug 09 15:38:09 dist-test-slave-2w3w krb5kdc[16078](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786289889, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.89.254@KRBTEST.COM for HTTP/127.3.89.212@KRBTEST.COM
I20260809 15:38:09.919673 17105 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.002s sys 0.001s
I20260809 15:38:09.921869 17121 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 15:38:09.923108 17105 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260809 15:38:09.923261 17105 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786289732823074-3431-0/minicluster-data/master-0/data,/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786289732823074-3431-0/minicluster-data/master-0/wal
uuid: "e056d1c2456c4d4287b5ca44155ff2f2"
format_stamp: "Formatted at 2026-08-09 15:38:07 on dist-test-slave-2w3w"
server_key: "50d6a8d215c64ee018ec656030f12b05"
server_key_iv: "2d4490c715c62af1278c805334c53eab"
server_key_version: "kuduclusterkey@0"
I20260809 15:38:09.923799 17105 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786289732823074-3431-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786289732823074-3431-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786289732823074-3431-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 15:38:09.935642 17105 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 15:38:09.938086 17105 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 15:38:09.938315 17105 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 15:38:09.945353 17105 rpc_server.cc:307] RPC server started. Bound to: 127.3.89.254:33879
I20260809 15:38:09.945359 17182 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.3.89.254:33879 every 8 connection(s)
I20260809 15:38:09.946415 17105 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786289732823074-3431-0/minicluster-data/master-0/data/info.pb
I20260809 15:38:09.950462 17183 sys_catalog.cc:263] Verifying existing consensus state
I20260809 15:38:09.951267 3431 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskm314YR/build/debug/bin/kudu as pid 17105
I20260809 15:38:09.951905 3431 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskm314YR/build/debug/bin/kudu
/tmp/dist-test-taskm314YR/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786289732823074-3431-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786289732823074-3431-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786289732823074-3431-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786289732823074-3431-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.3.89.212:58915/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.3.89.193:37409
--local_ip_for_outbound_sockets=127.3.89.193
--tserver_master_addrs=127.3.89.254:33879
--webserver_port=34171
--webserver_interface=127.3.89.193
--keytab_file=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786289732823074-3431-0/krb5kdc/kudu_127.3.89.193.keytab
--principal=kudu/127.3.89.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.3.89.212:35415
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--ranger_kms_url=invalid.host:1234/kms,127.3.89.212:58915/kms with env {KRB5CCNAME=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786289732823074-3431-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786289732823074-3431-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786289732823074-3431-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260809 15:38:09.952504 17183 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e056d1c2456c4d4287b5ca44155ff2f2: Bootstrap starting.
I20260809 15:38:09.969077 17183 log.cc:824] T 00000000000000000000000000000000 P e056d1c2456c4d4287b5ca44155ff2f2: Log is configured to *not* fsync() on all Append() calls
I20260809 15:38:09.975756 17183 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e056d1c2456c4d4287b5ca44155ff2f2: Bootstrap replayed 1/1 log segments. Stats: ops{read=4 overwritten=0 applied=4 ignored=0} inserts{seen=3 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260809 15:38:09.976181 17183 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e056d1c2456c4d4287b5ca44155ff2f2: Bootstrap complete.
I20260809 15:38:09.980389 17183 raft_consensus.cc:348] T 00000000000000000000000000000000 P e056d1c2456c4d4287b5ca44155ff2f2 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e056d1c2456c4d4287b5ca44155ff2f2" member_type: VOTER last_known_addr { host: "127.3.89.254" port: 33879 } }
I20260809 15:38:09.980788 17183 raft_consensus.cc:729] T 00000000000000000000000000000000 P e056d1c2456c4d4287b5ca44155ff2f2 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: e056d1c2456c4d4287b5ca44155ff2f2, State: Initialized, Role: FOLLOWER
I20260809 15:38:09.981331 17183 consensus_queue.cc:260] T 00000000000000000000000000000000 P e056d1c2456c4d4287b5ca44155ff2f2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e056d1c2456c4d4287b5ca44155ff2f2" member_type: VOTER last_known_addr { host: "127.3.89.254" port: 33879 } }
I20260809 15:38:09.981451 17183 raft_consensus.cc:388] T 00000000000000000000000000000000 P e056d1c2456c4d4287b5ca44155ff2f2 [term 1 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 15:38:09.981541 17183 raft_consensus.cc:482] T 00000000000000000000000000000000 P e056d1c2456c4d4287b5ca44155ff2f2 [term 1 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 15:38:09.981638 17183 raft_consensus.cc:3037] T 00000000000000000000000000000000 P e056d1c2456c4d4287b5ca44155ff2f2 [term 1 FOLLOWER]: Advancing to term 2
I20260809 15:38:09.983608 17183 raft_consensus.cc:504] T 00000000000000000000000000000000 P e056d1c2456c4d4287b5ca44155ff2f2 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e056d1c2456c4d4287b5ca44155ff2f2" member_type: VOTER last_known_addr { host: "127.3.89.254" port: 33879 } }
I20260809 15:38:09.983946 17183 leader_election.cc:302] T 00000000000000000000000000000000 P e056d1c2456c4d4287b5ca44155ff2f2 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: e056d1c2456c4d4287b5ca44155ff2f2; no voters:
I20260809 15:38:09.984328 17183 leader_election.cc:288] T 00000000000000000000000000000000 P e056d1c2456c4d4287b5ca44155ff2f2 [CANDIDATE]: Term 2 election: Requested vote from peers
I20260809 15:38:09.984397 17187 raft_consensus.cc:2781] T 00000000000000000000000000000000 P e056d1c2456c4d4287b5ca44155ff2f2 [term 2 FOLLOWER]: Leader election won for term 2
I20260809 15:38:09.984678 17187 raft_consensus.cc:686] T 00000000000000000000000000000000 P e056d1c2456c4d4287b5ca44155ff2f2 [term 2 LEADER]: Becoming Leader. State: Replica: e056d1c2456c4d4287b5ca44155ff2f2, State: Running, Role: LEADER
I20260809 15:38:09.985020 17187 consensus_queue.cc:237] T 00000000000000000000000000000000 P e056d1c2456c4d4287b5ca44155ff2f2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e056d1c2456c4d4287b5ca44155ff2f2" member_type: VOTER last_known_addr { host: "127.3.89.254" port: 33879 } }
I20260809 15:38:09.985558 17183 sys_catalog.cc:564] T 00000000000000000000000000000000 P e056d1c2456c4d4287b5ca44155ff2f2 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 15:38:09.986384 17188 sys_catalog.cc:455] T 00000000000000000000000000000000 P e056d1c2456c4d4287b5ca44155ff2f2 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 2 leader_uuid: "e056d1c2456c4d4287b5ca44155ff2f2" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e056d1c2456c4d4287b5ca44155ff2f2" member_type: VOTER last_known_addr { host: "127.3.89.254" port: 33879 } } }
I20260809 15:38:09.986544 17188 sys_catalog.cc:458] T 00000000000000000000000000000000 P e056d1c2456c4d4287b5ca44155ff2f2 [sys.catalog]: This master's current role is: LEADER
I20260809 15:38:09.987330 17189 sys_catalog.cc:455] T 00000000000000000000000000000000 P e056d1c2456c4d4287b5ca44155ff2f2 [sys.catalog]: SysCatalogTable state changed. Reason: New leader e056d1c2456c4d4287b5ca44155ff2f2. Latest consensus state: current_term: 2 leader_uuid: "e056d1c2456c4d4287b5ca44155ff2f2" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e056d1c2456c4d4287b5ca44155ff2f2" member_type: VOTER last_known_addr { host: "127.3.89.254" port: 33879 } } }
I20260809 15:38:09.987406 17196 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 15:38:09.987423 17189 sys_catalog.cc:458] T 00000000000000000000000000000000 P e056d1c2456c4d4287b5ca44155ff2f2 [sys.catalog]: This master's current role is: LEADER
I20260809 15:38:09.987972 17183 ranger_client.cc:318] Using overwritten properties file: /tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786289732823074-3431-0/minicluster-data/master-0/logs/kudu-ranger-subprocess-log4j2.properties
I20260809 15:38:09.993638 17196 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 15:38:09.994668 17196 catalog_manager.cc:1259] Loaded cluster ID: 160fa241659546dc8e93ecac319a6270
I20260809 15:38:09.994796 17196 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 15:38:09.997773 17196 catalog_manager.cc:1504] Loading token signing keys...
I20260809 15:38:09.998761 17196 catalog_manager.cc:5879] T 00000000000000000000000000000000 P e056d1c2456c4d4287b5ca44155ff2f2: Loaded TSK: 0
I20260809 15:38:10.000156 17196 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260809 15:38:10.093210 17185 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 15:38:10.093561 17185 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 15:38:10.093626 17185 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 15:38:10.099684 17185 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 15:38:10.099804 17185 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 15:38:10.099929 17185 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.3.89.193
I20260809 15:38:10.107510 17185 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.3.89.212:35415
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786289732823074-3431-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786289732823074-3431-0/minicluster-data/ts-0/wal
--ranger_kms_url=invalid.host:1234/kms,127.3.89.212:58915/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786289732823074-3431-0/krb5kdc/kudu_127.3.89.193.keytab
--principal=kudu/127.3.89.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.3.89.193:37409
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786289732823074-3431-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.3.89.193
--webserver_port=34171
--tserver_master_addrs=127.3.89.254:33879
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.17185
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.3.89.193
--log_dir=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786289732823074-3431-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 09 Aug 2026 14:46:55 UTC on ccycloud.quasar-yswkmf.root.comops.site
build id 4576540
I20260809 15:38:10.108785 17185 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 15:38:10.110024 17185 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 15:38:10.123154 17225 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 15:38:10.123274 17224 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 15:38:10.123167 17227 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 15:38:10.123654 17185 server_base.cc:1034] running on GCE node
I20260809 15:38:10.124094 17185 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 15:38:10.124923 17185 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 15:38:10.128252 17185 hybrid_clock.cc:648] HybridClock initialized: now 1786289890128215 us; error 49 us; skew 500 ppm
Aug 09 15:38:10 dist-test-slave-2w3w krb5kdc[16078](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786289890, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.89.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 15:38:10.131920 17185 init.cc:377] Logged in from keytab as kudu/127.3.89.193@KRBTEST.COM (short username kudu)
I20260809 15:38:10.132992 17185 webserver.cc:460] Webserver started at http://127.3.89.193:34171/ using document root <none> and password file <none>
I20260809 15:38:10.133708 17185 fs_manager.cc:359] Metadata directory not provided
I20260809 15:38:10.133809 17185 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
Aug 09 15:38:10 dist-test-slave-2w3w krb5kdc[16078](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786289890, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.89.193@KRBTEST.COM for HTTP/127.3.89.212@KRBTEST.COM
I20260809 15:38:10.172446 17185 fs_manager.cc:709] Time spent opening directory manager: real 0.014s user 0.000s sys 0.003s
I20260809 15:38:10.191980 17236 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 15:38:10.198117 17185 fs_manager.cc:725] Time spent opening block manager: real 0.024s user 0.002s sys 0.000s
I20260809 15:38:10.198282 17185 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786289732823074-3431-0/minicluster-data/ts-0/data,/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786289732823074-3431-0/minicluster-data/ts-0/wal
uuid: "28b33706caad4d379f73b37cbf0656c8"
format_stamp: "Formatted at 2026-08-09 15:38:08 on dist-test-slave-2w3w"
server_key: "20b284a3ff0e806bc9af0c499a84de90"
server_key_iv: "6cacfa3671ac7d86abcbe5c4ece48628"
server_key_version: "kuduclusterkey@0"
I20260809 15:38:10.199088 17185 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786289732823074-3431-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786289732823074-3431-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786289732823074-3431-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 15:38:10.215535 17185 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 15:38:10.219165 17185 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 15:38:10.219374 17185 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 15:38:10.220090 17185 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 15:38:10.221149 17185 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 15:38:10.221215 17185 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 15:38:10.221274 17185 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 15:38:10.221304 17185 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 15:38:10.232213 17185 rpc_server.cc:307] RPC server started. Bound to: 127.3.89.193:37409
I20260809 15:38:10.232461 17353 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.3.89.193:37409 every 8 connection(s)
I20260809 15:38:10.233417 17185 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786289732823074-3431-0/minicluster-data/ts-0/data/info.pb
I20260809 15:38:10.242702 3431 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskm314YR/build/debug/bin/kudu as pid 17185
I20260809 15:38:10.243275 3431 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskm314YR/build/debug/bin/kudu
/tmp/dist-test-taskm314YR/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786289732823074-3431-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786289732823074-3431-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786289732823074-3431-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786289732823074-3431-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.3.89.212:58915/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.3.89.194:39621
--local_ip_for_outbound_sockets=127.3.89.194
--tserver_master_addrs=127.3.89.254:33879
--webserver_port=39909
--webserver_interface=127.3.89.194
--keytab_file=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786289732823074-3431-0/krb5kdc/kudu_127.3.89.194.keytab
--principal=kudu/127.3.89.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.3.89.212:35415
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--ranger_kms_url=invalid.host:1234/kms,127.3.89.212:58915/kms with env {KRB5CCNAME=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786289732823074-3431-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786289732823074-3431-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786289732823074-3431-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 09 15:38:10 dist-test-slave-2w3w krb5kdc[16078](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786289890, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.89.193@KRBTEST.COM for kudu/127.3.89.254@KRBTEST.COM
I20260809 15:38:10.252180 17357 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:38:10.236934 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.89.193:33963 (local address 127.3.89.254:33879)
0809 15:38:10.237950 (+ 1016us) server_negotiation.cc:206] Beginning negotiation
0809 15:38:10.237972 (+ 22us) server_negotiation.cc:399] Waiting for connection header
0809 15:38:10.238016 (+ 44us) server_negotiation.cc:407] Connection header received
0809 15:38:10.238306 (+ 290us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 15:38:10.238322 (+ 16us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 15:38:10.238742 (+ 420us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 15:38:10.239128 (+ 386us) server_negotiation.cc:226] Negotiated authn=SASL
0809 15:38:10.240428 (+ 1300us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:38:10.242018 (+ 1590us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:38:10.243570 (+ 1552us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:38:10.244242 (+ 672us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:38:10.244429 (+ 187us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:38:10.246941 (+ 2512us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 15:38:10.246948 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 15:38:10.246987 (+ 39us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 15:38:10.247019 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0809 15:38:10.248886 (+ 1867us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 15:38:10.249445 (+ 559us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 15:38:10.249450 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 15:38:10.249455 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0809 15:38:10.249576 (+ 121us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 15:38:10.249928 (+ 352us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 15:38:10.249933 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 15:38:10.249935 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 15:38:10.250357 (+ 422us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 15:38:10.250507 (+ 150us) server_negotiation.cc:1035] Waiting for connection context
0809 15:38:10.251055 (+ 548us) server_negotiation.cc:299] Negotiation successful
0809 15:38:10.251296 (+ 241us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":861,"thread_start_us":116,"threads_started":1}
I20260809 15:38:10.252192 17356 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:38:10.235668 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.89.254:33879 (local address 127.3.89.193:33963)
0809 15:38:10.236212 (+ 544us) negotiation.cc:107] Waiting for socket to connect
0809 15:38:10.236296 (+ 84us) client_negotiation.cc:174] Beginning negotiation
0809 15:38:10.237447 (+ 1151us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 15:38:10.239240 (+ 1793us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 15:38:10.239252 (+ 12us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 15:38:10.239590 (+ 338us) client_negotiation.cc:189] Negotiated authn=SASL
0809 15:38:10.240229 (+ 639us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:38:10.240250 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:38:10.242176 (+ 1926us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:38:10.242180 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:38:10.243414 (+ 1234us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:38:10.243429 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:38:10.244415 (+ 986us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:38:10.244420 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:38:10.244615 (+ 195us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:38:10.245234 (+ 619us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 15:38:10.245250 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0809 15:38:10.246741 (+ 1491us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 15:38:10.249037 (+ 2296us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 15:38:10.249043 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 15:38:10.249055 (+ 12us) client_negotiation.cc:760] calling sasl_client_step()
0809 15:38:10.249312 (+ 257us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 15:38:10.249681 (+ 369us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 15:38:10.249686 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 15:38:10.249688 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 15:38:10.249825 (+ 137us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 15:38:10.250483 (+ 658us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 15:38:10.250489 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 15:38:10.250720 (+ 231us) client_negotiation.cc:769] Sending connection context
0809 15:38:10.250904 (+ 184us) client_negotiation.cc:240] Negotiation successful
0809 15:38:10.251235 (+ 331us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":318,"thread_start_us":124,"threads_started":1}
I20260809 15:38:10.255024 17354 heartbeater.cc:347] Connected to a master server at 127.3.89.254:33879
I20260809 15:38:10.255209 17354 heartbeater.cc:464] Registering TS with master...
I20260809 15:38:10.255749 17354 heartbeater.cc:511] Master 127.3.89.254:33879 requested a full tablet report, sending...
W20260809 15:38:10.258020 17354 heartbeater.cc:644] Failed to heartbeat to 127.3.89.254:33879 (0 consecutive failures): Service unavailable: Catalog manager is not initialized. State: Starting
I20260809 15:38:10.258142 17354 heartbeater.cc:464] Registering TS with master...
I20260809 15:38:10.258304 17354 heartbeater.cc:511] Master 127.3.89.254:33879 requested a full tablet report, sending...
I20260809 15:38:10.259137 17354 heartbeater.cc:464] Registering TS with master...
I20260809 15:38:10.259330 17354 heartbeater.cc:511] Master 127.3.89.254:33879 requested a full tablet report, sending...
W20260809 15:38:10.260110 17354 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260809 15:38:10.402309 17358 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 15:38:10.402648 17358 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 15:38:10.402724 17358 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 15:38:10.408797 17358 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 15:38:10.408880 17358 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 15:38:10.408968 17358 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.3.89.194
I20260809 15:38:10.416692 17358 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.3.89.212:35415
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786289732823074-3431-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786289732823074-3431-0/minicluster-data/ts-1/wal
--ranger_kms_url=invalid.host:1234/kms,127.3.89.212:58915/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786289732823074-3431-0/krb5kdc/kudu_127.3.89.194.keytab
--principal=kudu/127.3.89.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.3.89.194:39621
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786289732823074-3431-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.3.89.194
--webserver_port=39909
--tserver_master_addrs=127.3.89.254:33879
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.17358
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.3.89.194
--log_dir=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786289732823074-3431-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 09 Aug 2026 14:46:55 UTC on ccycloud.quasar-yswkmf.root.comops.site
build id 4576540
I20260809 15:38:10.418092 17358 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 15:38:10.419441 17358 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 15:38:10.432430 17367 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 15:38:10.432547 17364 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 15:38:10.432497 17365 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 15:38:10.433917 17358 server_base.cc:1034] running on GCE node
I20260809 15:38:10.434454 17358 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 15:38:10.435252 17358 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 15:38:10.436479 17358 hybrid_clock.cc:648] HybridClock initialized: now 1786289890436458 us; error 37 us; skew 500 ppm
Aug 09 15:38:10 dist-test-slave-2w3w krb5kdc[16078](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786289890, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.89.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 15:38:10.440182 17358 init.cc:377] Logged in from keytab as kudu/127.3.89.194@KRBTEST.COM (short username kudu)
I20260809 15:38:10.441286 17358 webserver.cc:460] Webserver started at http://127.3.89.194:39909/ using document root <none> and password file <none>
I20260809 15:38:10.441994 17358 fs_manager.cc:359] Metadata directory not provided
I20260809 15:38:10.442097 17358 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
Aug 09 15:38:10 dist-test-slave-2w3w krb5kdc[16078](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786289890, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.89.194@KRBTEST.COM for HTTP/127.3.89.212@KRBTEST.COM
I20260809 15:38:10.466629 17358 fs_manager.cc:709] Time spent opening directory manager: real 0.007s user 0.002s sys 0.000s
I20260809 15:38:10.469699 17375 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 15:38:10.471164 17358 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.000s sys 0.002s
I20260809 15:38:10.471312 17358 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786289732823074-3431-0/minicluster-data/ts-1/data,/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786289732823074-3431-0/minicluster-data/ts-1/wal
uuid: "e7f918425ade4bcbb5ff5f6ece608c08"
format_stamp: "Formatted at 2026-08-09 15:38:09 on dist-test-slave-2w3w"
server_key: "7b630996cc7dc8590df4d797011a3859"
server_key_iv: "b7e25b0825ec43744a7dddf88350d69d"
server_key_version: "kuduclusterkey@0"
I20260809 15:38:10.471886 17358 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786289732823074-3431-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786289732823074-3431-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786289732823074-3431-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 15:38:10.488579 17358 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 15:38:10.492300 17358 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 15:38:10.492522 17358 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 15:38:10.493235 17358 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 15:38:10.494278 17358 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 15:38:10.494362 17358 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 15:38:10.494427 17358 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 15:38:10.494462 17358 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 15:38:10.506073 17358 rpc_server.cc:307] RPC server started. Bound to: 127.3.89.194:39621
I20260809 15:38:10.506170 17499 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.3.89.194:39621 every 8 connection(s)
I20260809 15:38:10.507229 17358 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786289732823074-3431-0/minicluster-data/ts-1/data/info.pb
I20260809 15:38:10.514132 3431 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskm314YR/build/debug/bin/kudu as pid 17358
I20260809 15:38:10.514631 3431 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskm314YR/build/debug/bin/kudu
/tmp/dist-test-taskm314YR/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786289732823074-3431-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786289732823074-3431-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786289732823074-3431-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786289732823074-3431-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.3.89.212:58915/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.3.89.195:42979
--local_ip_for_outbound_sockets=127.3.89.195
--tserver_master_addrs=127.3.89.254:33879
--webserver_port=36985
--webserver_interface=127.3.89.195
--keytab_file=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786289732823074-3431-0/krb5kdc/kudu_127.3.89.195.keytab
--principal=kudu/127.3.89.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.3.89.212:35415
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--ranger_kms_url=invalid.host:1234/kms,127.3.89.212:58915/kms with env {KRB5CCNAME=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786289732823074-3431-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786289732823074-3431-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786289732823074-3431-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 09 15:38:10 dist-test-slave-2w3w krb5kdc[16078](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786289890, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.89.194@KRBTEST.COM for kudu/127.3.89.254@KRBTEST.COM
I20260809 15:38:10.523897 17357 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:38:10.510974 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.89.194:56789 (local address 127.3.89.254:33879)
0809 15:38:10.511135 (+ 161us) server_negotiation.cc:206] Beginning negotiation
0809 15:38:10.511140 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 15:38:10.511338 (+ 198us) server_negotiation.cc:407] Connection header received
0809 15:38:10.512493 (+ 1155us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 15:38:10.512497 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 15:38:10.512550 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 15:38:10.512646 (+ 96us) server_negotiation.cc:226] Negotiated authn=SASL
0809 15:38:10.514126 (+ 1480us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:38:10.515030 (+ 904us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:38:10.516803 (+ 1773us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:38:10.517166 (+ 363us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:38:10.517262 (+ 96us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:38:10.519998 (+ 2736us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 15:38:10.520001 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 15:38:10.520004 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 15:38:10.520029 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0809 15:38:10.521592 (+ 1563us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 15:38:10.522141 (+ 549us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 15:38:10.522145 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 15:38:10.522146 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 15:38:10.522231 (+ 85us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 15:38:10.522597 (+ 366us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 15:38:10.522602 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 15:38:10.522604 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 15:38:10.522794 (+ 190us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 15:38:10.522911 (+ 117us) server_negotiation.cc:1035] Waiting for connection context
0809 15:38:10.523586 (+ 675us) server_negotiation.cc:299] Negotiation successful
0809 15:38:10.523729 (+ 143us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":68}
I20260809 15:38:10.527207 17502 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:38:10.510061 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.89.254:33879 (local address 127.3.89.194:56789)
0809 15:38:10.511154 (+ 1093us) negotiation.cc:107] Waiting for socket to connect
0809 15:38:10.511213 (+ 59us) client_negotiation.cc:174] Beginning negotiation
0809 15:38:10.512268 (+ 1055us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 15:38:10.512861 (+ 593us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 15:38:10.512870 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 15:38:10.513260 (+ 390us) client_negotiation.cc:189] Negotiated authn=SASL
0809 15:38:10.513912 (+ 652us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:38:10.513933 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:38:10.515194 (+ 1261us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:38:10.515197 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:38:10.516644 (+ 1447us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:38:10.516656 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:38:10.517473 (+ 817us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:38:10.517477 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:38:10.517674 (+ 197us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:38:10.518391 (+ 717us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 15:38:10.518407 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0809 15:38:10.519825 (+ 1418us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 15:38:10.521730 (+ 1905us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 15:38:10.521736 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 15:38:10.521750 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0809 15:38:10.522018 (+ 268us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 15:38:10.522343 (+ 325us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 15:38:10.522346 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 15:38:10.522349 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0809 15:38:10.522492 (+ 143us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 15:38:10.522908 (+ 416us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 15:38:10.522916 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 15:38:10.523264 (+ 348us) client_negotiation.cc:769] Sending connection context
0809 15:38:10.524374 (+ 1110us) client_negotiation.cc:240] Negotiation successful
0809 15:38:10.526054 (+ 1680us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":848,"thread_start_us":132,"threads_started":1}
I20260809 15:38:10.528923 17500 heartbeater.cc:347] Connected to a master server at 127.3.89.254:33879
I20260809 15:38:10.529196 17500 heartbeater.cc:464] Registering TS with master...
I20260809 15:38:10.529763 17500 heartbeater.cc:511] Master 127.3.89.254:33879 requested a full tablet report, sending...
W20260809 15:38:10.531620 17500 heartbeater.cc:644] Failed to heartbeat to 127.3.89.254:33879 (0 consecutive failures): Service unavailable: Catalog manager is not initialized. State: Starting
I20260809 15:38:10.531744 17500 heartbeater.cc:464] Registering TS with master...
I20260809 15:38:10.531898 17500 heartbeater.cc:511] Master 127.3.89.254:33879 requested a full tablet report, sending...
I20260809 15:38:10.532853 17500 heartbeater.cc:464] Registering TS with master...
I20260809 15:38:10.533026 17500 heartbeater.cc:511] Master 127.3.89.254:33879 requested a full tablet report, sending...
W20260809 15:38:10.533908 17500 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260809 15:38:10.647936 17504 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 15:38:10.648200 17504 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 15:38:10.648303 17504 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 15:38:10.652601 17504 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 15:38:10.652674 17504 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 15:38:10.652787 17504 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.3.89.195
I20260809 15:38:10.658083 17504 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.3.89.212:35415
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786289732823074-3431-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786289732823074-3431-0/minicluster-data/ts-2/wal
--ranger_kms_url=invalid.host:1234/kms,127.3.89.212:58915/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786289732823074-3431-0/krb5kdc/kudu_127.3.89.195.keytab
--principal=kudu/127.3.89.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.3.89.195:42979
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786289732823074-3431-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.3.89.195
--webserver_port=36985
--tserver_master_addrs=127.3.89.254:33879
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.17504
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.3.89.195
--log_dir=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786289732823074-3431-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 09 Aug 2026 14:46:55 UTC on ccycloud.quasar-yswkmf.root.comops.site
build id 4576540
I20260809 15:38:10.659166 17504 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 15:38:10.660279 17504 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 15:38:10.670240 17511 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 15:38:10.670265 17510 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 15:38:10.670219 17513 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 15:38:10.671079 17504 server_base.cc:1034] running on GCE node
I20260809 15:38:10.671509 17504 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 15:38:10.672223 17504 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 15:38:10.675577 17504 hybrid_clock.cc:648] HybridClock initialized: now 1786289890675538 us; error 41 us; skew 500 ppm
Aug 09 15:38:10 dist-test-slave-2w3w krb5kdc[16078](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786289890, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.89.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 15:38:10.679349 17504 init.cc:377] Logged in from keytab as kudu/127.3.89.195@KRBTEST.COM (short username kudu)
I20260809 15:38:10.680469 17504 webserver.cc:460] Webserver started at http://127.3.89.195:36985/ using document root <none> and password file <none>
I20260809 15:38:10.681149 17504 fs_manager.cc:359] Metadata directory not provided
I20260809 15:38:10.681250 17504 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
Aug 09 15:38:10 dist-test-slave-2w3w krb5kdc[16078](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786289890, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.89.195@KRBTEST.COM for HTTP/127.3.89.212@KRBTEST.COM
I20260809 15:38:10.697850 17504 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.003s sys 0.000s
I20260809 15:38:10.701000 17522 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 15:38:10.702589 17504 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260809 15:38:10.702729 17504 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786289732823074-3431-0/minicluster-data/ts-2/data,/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786289732823074-3431-0/minicluster-data/ts-2/wal
uuid: "216a25fe11b84aed82d3183ea221a328"
format_stamp: "Formatted at 2026-08-09 15:38:09 on dist-test-slave-2w3w"
server_key: "9ec81d0949be86d130f1070f272a8623"
server_key_iv: "317f12e9197a3b430bd33fedd0632ba2"
server_key_version: "kuduclusterkey@0"
I20260809 15:38:10.703616 17504 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786289732823074-3431-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786289732823074-3431-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786289732823074-3431-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 15:38:10.730681 17504 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 15:38:10.734274 17504 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 15:38:10.734475 17504 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 15:38:10.735200 17504 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 15:38:10.736389 17504 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 15:38:10.736445 17504 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 15:38:10.736496 17504 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 15:38:10.736517 17504 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 15:38:10.747754 17504 rpc_server.cc:307] RPC server started. Bound to: 127.3.89.195:42979
I20260809 15:38:10.747910 17645 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.3.89.195:42979 every 8 connection(s)
I20260809 15:38:10.748840 17504 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786289732823074-3431-0/minicluster-data/ts-2/data/info.pb
I20260809 15:38:10.753934 3431 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskm314YR/build/debug/bin/kudu as pid 17504
Aug 09 15:38:10 dist-test-slave-2w3w krb5kdc[16078](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786289890, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.89.195@KRBTEST.COM for kudu/127.3.89.254@KRBTEST.COM
I20260809 15:38:10.766646 17357 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:38:10.750846 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.89.195:57411 (local address 127.3.89.254:33879)
0809 15:38:10.751035 (+ 189us) server_negotiation.cc:206] Beginning negotiation
0809 15:38:10.751040 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 15:38:10.751896 (+ 856us) server_negotiation.cc:407] Connection header received
0809 15:38:10.753120 (+ 1224us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 15:38:10.753124 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 15:38:10.753179 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 15:38:10.753272 (+ 93us) server_negotiation.cc:226] Negotiated authn=SASL
0809 15:38:10.754756 (+ 1484us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:38:10.755628 (+ 872us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:38:10.757394 (+ 1766us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:38:10.757721 (+ 327us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:38:10.757824 (+ 103us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:38:10.762185 (+ 4361us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 15:38:10.762191 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 15:38:10.762197 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 15:38:10.762228 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0809 15:38:10.764105 (+ 1877us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 15:38:10.764741 (+ 636us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 15:38:10.764745 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 15:38:10.764747 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 15:38:10.764865 (+ 118us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 15:38:10.765282 (+ 417us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 15:38:10.765286 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 15:38:10.765287 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 15:38:10.765511 (+ 224us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 15:38:10.765621 (+ 110us) server_negotiation.cc:1035] Waiting for connection context
0809 15:38:10.766297 (+ 676us) server_negotiation.cc:299] Negotiation successful
0809 15:38:10.766452 (+ 155us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":74}
I20260809 15:38:10.769479 17648 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:38:10.751211 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.89.254:33879 (local address 127.3.89.195:57411)
0809 15:38:10.751700 (+ 489us) negotiation.cc:107] Waiting for socket to connect
0809 15:38:10.751759 (+ 59us) client_negotiation.cc:174] Beginning negotiation
0809 15:38:10.752869 (+ 1110us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 15:38:10.753492 (+ 623us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 15:38:10.753502 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 15:38:10.753887 (+ 385us) client_negotiation.cc:189] Negotiated authn=SASL
0809 15:38:10.754542 (+ 655us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:38:10.754565 (+ 23us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:38:10.755803 (+ 1238us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:38:10.755806 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:38:10.757195 (+ 1389us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:38:10.757208 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:38:10.757901 (+ 693us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:38:10.757905 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:38:10.758130 (+ 225us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:38:10.758855 (+ 725us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 15:38:10.758875 (+ 20us) client_negotiation.cc:656] calling sasl_client_start()
0809 15:38:10.761998 (+ 3123us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 15:38:10.764236 (+ 2238us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 15:38:10.764243 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 15:38:10.764258 (+ 15us) client_negotiation.cc:760] calling sasl_client_step()
0809 15:38:10.764623 (+ 365us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 15:38:10.764987 (+ 364us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 15:38:10.764991 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 15:38:10.764993 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 15:38:10.765183 (+ 190us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 15:38:10.765645 (+ 462us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 15:38:10.765652 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 15:38:10.765968 (+ 316us) client_negotiation.cc:769] Sending connection context
0809 15:38:10.767963 (+ 1995us) client_negotiation.cc:240] Negotiation successful
0809 15:38:10.768357 (+ 394us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":279,"thread_start_us":111,"threads_started":1}
I20260809 15:38:10.770745 17646 heartbeater.cc:347] Connected to a master server at 127.3.89.254:33879
I20260809 15:38:10.771035 17646 heartbeater.cc:464] Registering TS with master...
I20260809 15:38:10.771644 17646 heartbeater.cc:511] Master 127.3.89.254:33879 requested a full tablet report, sending...
I20260809 15:38:10.773128 17140 ts_manager.cc:194] Registered new tserver with Master: 216a25fe11b84aed82d3183ea221a328 (127.3.89.195:42979)
I20260809 15:38:10.774672 17140 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.3.89.195@KRBTEST.COM'} at 127.3.89.195:57411
I20260809 15:38:10.776553 17650 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:38:10.754579 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53602 (local address 127.3.89.254:33879)
0809 15:38:10.755134 (+ 555us) server_negotiation.cc:206] Beginning negotiation
0809 15:38:10.755137 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0809 15:38:10.755379 (+ 242us) server_negotiation.cc:407] Connection header received
0809 15:38:10.755542 (+ 163us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 15:38:10.755545 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 15:38:10.755595 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 15:38:10.755696 (+ 101us) server_negotiation.cc:226] Negotiated authn=SASL
0809 15:38:10.756652 (+ 956us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:38:10.757164 (+ 512us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:38:10.761804 (+ 4640us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:38:10.762133 (+ 329us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:38:10.762233 (+ 100us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:38:10.765638 (+ 3405us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 15:38:10.765643 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 15:38:10.765647 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 15:38:10.765672 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0809 15:38:10.767486 (+ 1814us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 15:38:10.767975 (+ 489us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 15:38:10.767978 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 15:38:10.767980 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 15:38:10.768070 (+ 90us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 15:38:10.770792 (+ 2722us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 15:38:10.770796 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 15:38:10.770798 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 15:38:10.771061 (+ 263us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 15:38:10.771232 (+ 171us) server_negotiation.cc:1035] Waiting for connection context
0809 15:38:10.776241 (+ 5009us) server_negotiation.cc:299] Negotiation successful
0809 15:38:10.776385 (+ 144us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":498,"thread_start_us":100,"threads_started":1}
Aug 09 15:38:11 dist-test-slave-2w3w krb5kdc[16078](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1786289891, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.89.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 15:38:11.260360 17354 heartbeater.cc:464] Registering TS with master...
I20260809 15:38:11.260658 17354 heartbeater.cc:511] Master 127.3.89.254:33879 requested a full tablet report, sending...
I20260809 15:38:11.261515 17140 ts_manager.cc:194] Registered new tserver with Master: 28b33706caad4d379f73b37cbf0656c8 (127.3.89.193:37409)
I20260809 15:38:11.262362 17140 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.3.89.193@KRBTEST.COM'} at 127.3.89.193:33963
Aug 09 15:38:11 dist-test-slave-2w3w krb5kdc[16078](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786289891, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.89.254@KRBTEST.COM for HTTP/127.3.89.212@KRBTEST.COM
I20260809 15:38:11.534169 17500 heartbeater.cc:464] Registering TS with master...
I20260809 15:38:11.534474 17500 heartbeater.cc:511] Master 127.3.89.254:33879 requested a full tablet report, sending...
I20260809 15:38:11.535280 17140 ts_manager.cc:194] Registered new tserver with Master: e7f918425ade4bcbb5ff5f6ece608c08 (127.3.89.194:39621)
I20260809 15:38:11.535997 17140 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.3.89.194@KRBTEST.COM'} at 127.3.89.194:56789
I20260809 15:38:11.545468 3431 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260809 15:38:11.555019 17671 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:38:11.547226 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53608 (local address 127.3.89.254:33879)
0809 15:38:11.547501 (+ 275us) server_negotiation.cc:206] Beginning negotiation
0809 15:38:11.547505 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 15:38:11.547524 (+ 19us) server_negotiation.cc:407] Connection header received
0809 15:38:11.547639 (+ 115us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 15:38:11.547642 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 15:38:11.547690 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 15:38:11.547791 (+ 101us) server_negotiation.cc:226] Negotiated authn=SASL
0809 15:38:11.548446 (+ 655us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:38:11.549222 (+ 776us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:38:11.550062 (+ 840us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:38:11.550375 (+ 313us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:38:11.550473 (+ 98us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:38:11.551798 (+ 1325us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 15:38:11.551802 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 15:38:11.551805 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 15:38:11.551830 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0809 15:38:11.553385 (+ 1555us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 15:38:11.553789 (+ 404us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 15:38:11.553792 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 15:38:11.553794 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 15:38:11.553881 (+ 87us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 15:38:11.554136 (+ 255us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 15:38:11.554139 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 15:38:11.554140 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 15:38:11.554331 (+ 191us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 15:38:11.554455 (+ 124us) server_negotiation.cc:1035] Waiting for connection context
0809 15:38:11.554663 (+ 208us) server_negotiation.cc:299] Negotiation successful
0809 15:38:11.554811 (+ 148us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":198,"thread_start_us":115,"threads_started":1}
I20260809 15:38:11.558876 17140 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:53608:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260809 15:38:11.563438 17140 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260809 15:38:11.581024 17677 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:38:11.573681 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.89.193:37409 (local address 127.0.0.1:49344)
0809 15:38:11.574948 (+ 1267us) negotiation.cc:107] Waiting for socket to connect
0809 15:38:11.575032 (+ 84us) client_negotiation.cc:174] Beginning negotiation
0809 15:38:11.575151 (+ 119us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 15:38:11.575950 (+ 799us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 15:38:11.575958 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 15:38:11.575986 (+ 28us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 15:38:11.576148 (+ 162us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:38:11.576157 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:38:11.577727 (+ 1570us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:38:11.577732 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:38:11.579129 (+ 1397us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:38:11.579142 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:38:11.580510 (+ 1368us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:38:11.580514 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:38:11.580617 (+ 103us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:38:11.580643 (+ 26us) client_negotiation.cc:769] Sending connection context
0809 15:38:11.580739 (+ 96us) client_negotiation.cc:240] Negotiation successful
0809 15:38:11.580838 (+ 99us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1092,"mutex_wait_us":1,"thread_start_us":115,"threads_started":1}
I20260809 15:38:11.580824 17676 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:38:11.573737 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.89.195:42979 (local address 127.0.0.1:40190)
0809 15:38:11.575313 (+ 1576us) negotiation.cc:107] Waiting for socket to connect
0809 15:38:11.575325 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0809 15:38:11.575484 (+ 159us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 15:38:11.576109 (+ 625us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 15:38:11.576113 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 15:38:11.576129 (+ 16us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 15:38:11.576221 (+ 92us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:38:11.576226 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:38:11.577563 (+ 1337us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:38:11.577566 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:38:11.579048 (+ 1482us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:38:11.579056 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:38:11.580254 (+ 1198us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:38:11.580257 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:38:11.580367 (+ 110us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:38:11.580405 (+ 38us) client_negotiation.cc:769] Sending connection context
0809 15:38:11.580509 (+ 104us) client_negotiation.cc:240] Negotiation successful
0809 15:38:11.580638 (+ 129us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1516,"mutex_wait_us":47,"thread_start_us":110,"threads_started":1}
I20260809 15:38:11.581633 17681 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:38:11.574449 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40190 (local address 127.3.89.195:42979)
0809 15:38:11.574870 (+ 421us) server_negotiation.cc:206] Beginning negotiation
0809 15:38:11.574878 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0809 15:38:11.575424 (+ 546us) server_negotiation.cc:407] Connection header received
0809 15:38:11.575618 (+ 194us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 15:38:11.575623 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 15:38:11.575806 (+ 183us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 15:38:11.575961 (+ 155us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 15:38:11.576365 (+ 404us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:38:11.577446 (+ 1081us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:38:11.579166 (+ 1720us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:38:11.579947 (+ 781us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:38:11.580047 (+ 100us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:38:11.580199 (+ 152us) server_negotiation.cc:1035] Waiting for connection context
0809 15:38:11.580926 (+ 727us) server_negotiation.cc:299] Negotiation successful
0809 15:38:11.581033 (+ 107us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":261,"thread_start_us":128,"threads_started":1}
I20260809 15:38:11.581676 17680 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:38:11.574461 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49344 (local address 127.3.89.193:37409)
0809 15:38:11.575087 (+ 626us) server_negotiation.cc:206] Beginning negotiation
0809 15:38:11.575097 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0809 15:38:11.575126 (+ 29us) server_negotiation.cc:407] Connection header received
0809 15:38:11.575340 (+ 214us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 15:38:11.575349 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 15:38:11.575572 (+ 223us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 15:38:11.575742 (+ 170us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 15:38:11.576315 (+ 573us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:38:11.577581 (+ 1266us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:38:11.579310 (+ 1729us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:38:11.580187 (+ 877us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:38:11.580304 (+ 117us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:38:11.580441 (+ 137us) server_negotiation.cc:1035] Waiting for connection context
0809 15:38:11.581403 (+ 962us) server_negotiation.cc:299] Negotiation successful
0809 15:38:11.581513 (+ 110us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":487,"thread_start_us":65,"threads_started":1}
I20260809 15:38:11.582706 17678 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:38:11.573988 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.89.194:39621 (local address 127.0.0.1:52996)
0809 15:38:11.575946 (+ 1958us) negotiation.cc:107] Waiting for socket to connect
0809 15:38:11.575960 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0809 15:38:11.576045 (+ 85us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 15:38:11.576577 (+ 532us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 15:38:11.576579 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 15:38:11.576596 (+ 17us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 15:38:11.576676 (+ 80us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:38:11.576681 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:38:11.578964 (+ 2283us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:38:11.578967 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:38:11.580066 (+ 1099us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:38:11.580075 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:38:11.582326 (+ 2251us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:38:11.582330 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:38:11.582418 (+ 88us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:38:11.582440 (+ 22us) client_negotiation.cc:769] Sending connection context
0809 15:38:11.582514 (+ 74us) client_negotiation.cc:240] Negotiation successful
0809 15:38:11.582599 (+ 85us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1839,"thread_start_us":92,"threads_started":1}
I20260809 15:38:11.583184 17679 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:38:11.574154 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52996 (local address 127.3.89.194:39621)
0809 15:38:11.575752 (+ 1598us) server_negotiation.cc:206] Beginning negotiation
0809 15:38:11.575758 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 15:38:11.576148 (+ 390us) server_negotiation.cc:407] Connection header received
0809 15:38:11.576225 (+ 77us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 15:38:11.576230 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 15:38:11.576405 (+ 175us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 15:38:11.576802 (+ 397us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 15:38:11.576937 (+ 135us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:38:11.578788 (+ 1851us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:38:11.580191 (+ 1403us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:38:11.582051 (+ 1860us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:38:11.582150 (+ 99us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:38:11.582266 (+ 116us) server_negotiation.cc:1035] Waiting for connection context
0809 15:38:11.582937 (+ 671us) server_negotiation.cc:299] Negotiation successful
0809 15:38:11.583051 (+ 114us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1505,"thread_start_us":98,"threads_started":1}
I20260809 15:38:11.584630 17285 tablet_service.cc:1467] Processing CreateTablet for tablet 40303847baa04254bd3a55f2e59a7dde (DEFAULT_TABLE table=test-table [id=6600305929fa49dbb0c7f7bea58ccea8]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 15:38:11.584630 17576 tablet_service.cc:1467] Processing CreateTablet for tablet 40303847baa04254bd3a55f2e59a7dde (DEFAULT_TABLE table=test-table [id=6600305929fa49dbb0c7f7bea58ccea8]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 15:38:11.585722 17576 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 40303847baa04254bd3a55f2e59a7dde. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 15:38:11.585721 17285 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 40303847baa04254bd3a55f2e59a7dde. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 15:38:11.585683 17429 tablet_service.cc:1467] Processing CreateTablet for tablet 40303847baa04254bd3a55f2e59a7dde (DEFAULT_TABLE table=test-table [id=6600305929fa49dbb0c7f7bea58ccea8]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 15:38:11.586730 17429 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 40303847baa04254bd3a55f2e59a7dde. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 15:38:11.596177 17682 tablet_bootstrap.cc:492] T 40303847baa04254bd3a55f2e59a7dde P 216a25fe11b84aed82d3183ea221a328: Bootstrap starting.
I20260809 15:38:11.598565 17682 tablet_bootstrap.cc:654] T 40303847baa04254bd3a55f2e59a7dde P 216a25fe11b84aed82d3183ea221a328: Neither blocks nor log segments found. Creating new log.
I20260809 15:38:11.599483 17682 log.cc:824] T 40303847baa04254bd3a55f2e59a7dde P 216a25fe11b84aed82d3183ea221a328: Log is configured to *not* fsync() on all Append() calls
I20260809 15:38:11.601596 17682 tablet_bootstrap.cc:492] T 40303847baa04254bd3a55f2e59a7dde P 216a25fe11b84aed82d3183ea221a328: No bootstrap required, opened a new log
I20260809 15:38:11.601794 17682 ts_tablet_manager.cc:1397] T 40303847baa04254bd3a55f2e59a7dde P 216a25fe11b84aed82d3183ea221a328: Time spent bootstrapping tablet: real 0.006s user 0.005s sys 0.000s
I20260809 15:38:11.606307 17682 raft_consensus.cc:348] T 40303847baa04254bd3a55f2e59a7dde P 216a25fe11b84aed82d3183ea221a328 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "28b33706caad4d379f73b37cbf0656c8" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 37409 } } peers { permanent_uuid: "216a25fe11b84aed82d3183ea221a328" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 42979 } } peers { permanent_uuid: "e7f918425ade4bcbb5ff5f6ece608c08" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 39621 } }
I20260809 15:38:11.606567 17682 raft_consensus.cc:374] T 40303847baa04254bd3a55f2e59a7dde P 216a25fe11b84aed82d3183ea221a328 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 15:38:11.606608 17682 raft_consensus.cc:729] T 40303847baa04254bd3a55f2e59a7dde P 216a25fe11b84aed82d3183ea221a328 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 216a25fe11b84aed82d3183ea221a328, State: Initialized, Role: FOLLOWER
I20260809 15:38:11.607074 17682 consensus_queue.cc:260] T 40303847baa04254bd3a55f2e59a7dde P 216a25fe11b84aed82d3183ea221a328 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "28b33706caad4d379f73b37cbf0656c8" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 37409 } } peers { permanent_uuid: "216a25fe11b84aed82d3183ea221a328" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 42979 } } peers { permanent_uuid: "e7f918425ade4bcbb5ff5f6ece608c08" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 39621 } }
I20260809 15:38:11.607542 17685 tablet_bootstrap.cc:492] T 40303847baa04254bd3a55f2e59a7dde P 28b33706caad4d379f73b37cbf0656c8: Bootstrap starting.
I20260809 15:38:11.608060 17682 ts_tablet_manager.cc:1428] T 40303847baa04254bd3a55f2e59a7dde P 216a25fe11b84aed82d3183ea221a328: Time spent starting tablet: real 0.006s user 0.006s sys 0.000s
I20260809 15:38:11.608412 17646 heartbeater.cc:503] Master 127.3.89.254:33879 was elected leader, sending a full tablet report...
I20260809 15:38:11.610406 17685 tablet_bootstrap.cc:654] T 40303847baa04254bd3a55f2e59a7dde P 28b33706caad4d379f73b37cbf0656c8: Neither blocks nor log segments found. Creating new log.
I20260809 15:38:11.610697 17683 tablet_bootstrap.cc:492] T 40303847baa04254bd3a55f2e59a7dde P e7f918425ade4bcbb5ff5f6ece608c08: Bootstrap starting.
I20260809 15:38:11.613946 17685 log.cc:824] T 40303847baa04254bd3a55f2e59a7dde P 28b33706caad4d379f73b37cbf0656c8: Log is configured to *not* fsync() on all Append() calls
I20260809 15:38:11.614672 17683 tablet_bootstrap.cc:654] T 40303847baa04254bd3a55f2e59a7dde P e7f918425ade4bcbb5ff5f6ece608c08: Neither blocks nor log segments found. Creating new log.
I20260809 15:38:11.615563 17683 log.cc:824] T 40303847baa04254bd3a55f2e59a7dde P e7f918425ade4bcbb5ff5f6ece608c08: Log is configured to *not* fsync() on all Append() calls
I20260809 15:38:11.616142 17685 tablet_bootstrap.cc:492] T 40303847baa04254bd3a55f2e59a7dde P 28b33706caad4d379f73b37cbf0656c8: No bootstrap required, opened a new log
I20260809 15:38:11.616333 17685 ts_tablet_manager.cc:1397] T 40303847baa04254bd3a55f2e59a7dde P 28b33706caad4d379f73b37cbf0656c8: Time spent bootstrapping tablet: real 0.009s user 0.004s sys 0.000s
I20260809 15:38:11.620183 17685 raft_consensus.cc:348] T 40303847baa04254bd3a55f2e59a7dde P 28b33706caad4d379f73b37cbf0656c8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "28b33706caad4d379f73b37cbf0656c8" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 37409 } } peers { permanent_uuid: "216a25fe11b84aed82d3183ea221a328" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 42979 } } peers { permanent_uuid: "e7f918425ade4bcbb5ff5f6ece608c08" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 39621 } }
I20260809 15:38:11.620437 17685 raft_consensus.cc:374] T 40303847baa04254bd3a55f2e59a7dde P 28b33706caad4d379f73b37cbf0656c8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 15:38:11.620483 17685 raft_consensus.cc:729] T 40303847baa04254bd3a55f2e59a7dde P 28b33706caad4d379f73b37cbf0656c8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 28b33706caad4d379f73b37cbf0656c8, State: Initialized, Role: FOLLOWER
I20260809 15:38:11.620882 17685 consensus_queue.cc:260] T 40303847baa04254bd3a55f2e59a7dde P 28b33706caad4d379f73b37cbf0656c8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "28b33706caad4d379f73b37cbf0656c8" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 37409 } } peers { permanent_uuid: "216a25fe11b84aed82d3183ea221a328" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 42979 } } peers { permanent_uuid: "e7f918425ade4bcbb5ff5f6ece608c08" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 39621 } }
I20260809 15:38:11.620956 17683 tablet_bootstrap.cc:492] T 40303847baa04254bd3a55f2e59a7dde P e7f918425ade4bcbb5ff5f6ece608c08: No bootstrap required, opened a new log
I20260809 15:38:11.621235 17683 ts_tablet_manager.cc:1397] T 40303847baa04254bd3a55f2e59a7dde P e7f918425ade4bcbb5ff5f6ece608c08: Time spent bootstrapping tablet: real 0.012s user 0.005s sys 0.000s
I20260809 15:38:11.621788 17685 ts_tablet_manager.cc:1428] T 40303847baa04254bd3a55f2e59a7dde P 28b33706caad4d379f73b37cbf0656c8: Time spent starting tablet: real 0.005s user 0.006s sys 0.000s
I20260809 15:38:11.622102 17354 heartbeater.cc:503] Master 127.3.89.254:33879 was elected leader, sending a full tablet report...
I20260809 15:38:11.628082 17683 raft_consensus.cc:348] T 40303847baa04254bd3a55f2e59a7dde P e7f918425ade4bcbb5ff5f6ece608c08 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "28b33706caad4d379f73b37cbf0656c8" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 37409 } } peers { permanent_uuid: "216a25fe11b84aed82d3183ea221a328" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 42979 } } peers { permanent_uuid: "e7f918425ade4bcbb5ff5f6ece608c08" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 39621 } }
I20260809 15:38:11.628364 17683 raft_consensus.cc:374] T 40303847baa04254bd3a55f2e59a7dde P e7f918425ade4bcbb5ff5f6ece608c08 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 15:38:11.628445 17683 raft_consensus.cc:729] T 40303847baa04254bd3a55f2e59a7dde P e7f918425ade4bcbb5ff5f6ece608c08 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e7f918425ade4bcbb5ff5f6ece608c08, State: Initialized, Role: FOLLOWER
I20260809 15:38:11.628991 17683 consensus_queue.cc:260] T 40303847baa04254bd3a55f2e59a7dde P e7f918425ade4bcbb5ff5f6ece608c08 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "28b33706caad4d379f73b37cbf0656c8" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 37409 } } peers { permanent_uuid: "216a25fe11b84aed82d3183ea221a328" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 42979 } } peers { permanent_uuid: "e7f918425ade4bcbb5ff5f6ece608c08" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 39621 } }
I20260809 15:38:11.629635 17500 heartbeater.cc:503] Master 127.3.89.254:33879 was elected leader, sending a full tablet report...
I20260809 15:38:11.631121 17683 ts_tablet_manager.cc:1428] T 40303847baa04254bd3a55f2e59a7dde P e7f918425ade4bcbb5ff5f6ece608c08: Time spent starting tablet: real 0.010s user 0.006s sys 0.000s
Aug 09 15:38:11 dist-test-slave-2w3w krb5kdc[16078](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786289891, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.89.254@KRBTEST.COM for HTTP/127.3.89.212@KRBTEST.COM
W20260809 15:38:11.736364 17355 tablet.cc:2369] T 40303847baa04254bd3a55f2e59a7dde P 28b33706caad4d379f73b37cbf0656c8: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260809 15:38:11.750387 17647 tablet.cc:2369] T 40303847baa04254bd3a55f2e59a7dde P 216a25fe11b84aed82d3183ea221a328: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260809 15:38:11.759025 17501 tablet.cc:2369] T 40303847baa04254bd3a55f2e59a7dde P e7f918425ade4bcbb5ff5f6ece608c08: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260809 15:38:11.805387 17689 raft_consensus.cc:482] T 40303847baa04254bd3a55f2e59a7dde P 28b33706caad4d379f73b37cbf0656c8 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 15:38:11.805629 17689 raft_consensus.cc:504] T 40303847baa04254bd3a55f2e59a7dde P 28b33706caad4d379f73b37cbf0656c8 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "28b33706caad4d379f73b37cbf0656c8" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 37409 } } peers { permanent_uuid: "216a25fe11b84aed82d3183ea221a328" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 42979 } } peers { permanent_uuid: "e7f918425ade4bcbb5ff5f6ece608c08" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 39621 } }
I20260809 15:38:11.807304 17689 leader_election.cc:288] T 40303847baa04254bd3a55f2e59a7dde P 28b33706caad4d379f73b37cbf0656c8 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 216a25fe11b84aed82d3183ea221a328 (127.3.89.195:42979), e7f918425ade4bcbb5ff5f6ece608c08 (127.3.89.194:39621)
I20260809 15:38:11.814709 17679 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:38:11.808391 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.89.193:33223 (local address 127.3.89.194:39621)
0809 15:38:11.808929 (+ 538us) server_negotiation.cc:206] Beginning negotiation
0809 15:38:11.808934 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 15:38:11.810253 (+ 1319us) server_negotiation.cc:407] Connection header received
0809 15:38:11.810460 (+ 207us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 15:38:11.810465 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 15:38:11.810515 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 15:38:11.810559 (+ 44us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 15:38:11.810999 (+ 440us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:38:11.811756 (+ 757us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:38:11.813360 (+ 1604us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:38:11.813978 (+ 618us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:38:11.814077 (+ 99us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:38:11.814134 (+ 57us) server_negotiation.cc:1035] Waiting for connection context
0809 15:38:11.814496 (+ 362us) server_negotiation.cc:299] Negotiation successful
0809 15:38:11.814581 (+ 85us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":444}
I20260809 15:38:11.815421 17693 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:38:11.807904 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.89.194:39621 (local address 127.3.89.193:33223)
0809 15:38:11.810162 (+ 2258us) negotiation.cc:107] Waiting for socket to connect
0809 15:38:11.810178 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0809 15:38:11.810333 (+ 155us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 15:38:11.810690 (+ 357us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 15:38:11.810693 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 15:38:11.810708 (+ 15us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 15:38:11.810857 (+ 149us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:38:11.810862 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:38:11.811928 (+ 1066us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:38:11.811931 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:38:11.813212 (+ 1281us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:38:11.813225 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:38:11.814237 (+ 1012us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:38:11.814241 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:38:11.814340 (+ 99us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:38:11.814366 (+ 26us) client_negotiation.cc:769] Sending connection context
0809 15:38:11.815196 (+ 830us) client_negotiation.cc:240] Negotiation successful
0809 15:38:11.815278 (+ 82us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":2186,"thread_start_us":130,"threads_started":1}
I20260809 15:38:11.816234 17449 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "40303847baa04254bd3a55f2e59a7dde" candidate_uuid: "28b33706caad4d379f73b37cbf0656c8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e7f918425ade4bcbb5ff5f6ece608c08" is_pre_election: true
I20260809 15:38:11.816567 17449 raft_consensus.cc:2454] T 40303847baa04254bd3a55f2e59a7dde P e7f918425ade4bcbb5ff5f6ece608c08 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 28b33706caad4d379f73b37cbf0656c8 in term 0.
I20260809 15:38:11.817185 17237 leader_election.cc:302] T 40303847baa04254bd3a55f2e59a7dde P 28b33706caad4d379f73b37cbf0656c8 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 28b33706caad4d379f73b37cbf0656c8, e7f918425ade4bcbb5ff5f6ece608c08; no voters:
I20260809 15:38:11.817502 17689 raft_consensus.cc:2781] T 40303847baa04254bd3a55f2e59a7dde P 28b33706caad4d379f73b37cbf0656c8 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260809 15:38:11.817587 17689 raft_consensus.cc:482] T 40303847baa04254bd3a55f2e59a7dde P 28b33706caad4d379f73b37cbf0656c8 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 15:38:11.817636 17689 raft_consensus.cc:3037] T 40303847baa04254bd3a55f2e59a7dde P 28b33706caad4d379f73b37cbf0656c8 [term 0 FOLLOWER]: Advancing to term 1
I20260809 15:38:11.819173 17689 raft_consensus.cc:504] T 40303847baa04254bd3a55f2e59a7dde P 28b33706caad4d379f73b37cbf0656c8 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "28b33706caad4d379f73b37cbf0656c8" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 37409 } } peers { permanent_uuid: "216a25fe11b84aed82d3183ea221a328" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 42979 } } peers { permanent_uuid: "e7f918425ade4bcbb5ff5f6ece608c08" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 39621 } }
I20260809 15:38:11.819568 17689 leader_election.cc:288] T 40303847baa04254bd3a55f2e59a7dde P 28b33706caad4d379f73b37cbf0656c8 [CANDIDATE]: Term 1 election: Requested vote from peers 216a25fe11b84aed82d3183ea221a328 (127.3.89.195:42979), e7f918425ade4bcbb5ff5f6ece608c08 (127.3.89.194:39621)
I20260809 15:38:11.820202 17449 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "40303847baa04254bd3a55f2e59a7dde" candidate_uuid: "28b33706caad4d379f73b37cbf0656c8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e7f918425ade4bcbb5ff5f6ece608c08"
I20260809 15:38:11.820326 17449 raft_consensus.cc:3037] T 40303847baa04254bd3a55f2e59a7dde P e7f918425ade4bcbb5ff5f6ece608c08 [term 0 FOLLOWER]: Advancing to term 1
I20260809 15:38:11.821689 17449 raft_consensus.cc:2454] T 40303847baa04254bd3a55f2e59a7dde P e7f918425ade4bcbb5ff5f6ece608c08 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 28b33706caad4d379f73b37cbf0656c8 in term 1.
I20260809 15:38:11.822093 17237 leader_election.cc:302] T 40303847baa04254bd3a55f2e59a7dde P 28b33706caad4d379f73b37cbf0656c8 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 28b33706caad4d379f73b37cbf0656c8, e7f918425ade4bcbb5ff5f6ece608c08; no voters:
I20260809 15:38:11.822327 17689 raft_consensus.cc:2781] T 40303847baa04254bd3a55f2e59a7dde P 28b33706caad4d379f73b37cbf0656c8 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 15:38:11.822553 17689 raft_consensus.cc:686] T 40303847baa04254bd3a55f2e59a7dde P 28b33706caad4d379f73b37cbf0656c8 [term 1 LEADER]: Becoming Leader. State: Replica: 28b33706caad4d379f73b37cbf0656c8, State: Running, Role: LEADER
I20260809 15:38:11.822942 17689 consensus_queue.cc:237] T 40303847baa04254bd3a55f2e59a7dde P 28b33706caad4d379f73b37cbf0656c8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "28b33706caad4d379f73b37cbf0656c8" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 37409 } } peers { permanent_uuid: "216a25fe11b84aed82d3183ea221a328" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 42979 } } peers { permanent_uuid: "e7f918425ade4bcbb5ff5f6ece608c08" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 39621 } }
I20260809 15:38:11.826416 17140 catalog_manager.cc:5495] T 40303847baa04254bd3a55f2e59a7dde P 28b33706caad4d379f73b37cbf0656c8 reported cstate change: term changed from 0 to 1, leader changed from <none> to 28b33706caad4d379f73b37cbf0656c8 (127.3.89.193). New cstate: current_term: 1 leader_uuid: "28b33706caad4d379f73b37cbf0656c8" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "28b33706caad4d379f73b37cbf0656c8" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 37409 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "216a25fe11b84aed82d3183ea221a328" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 42979 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e7f918425ade4bcbb5ff5f6ece608c08" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 39621 } health_report { overall_health: UNKNOWN } } }
I20260809 15:38:11.830323 17681 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:38:11.809022 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.89.193:40477 (local address 127.3.89.195:42979)
0809 15:38:11.809142 (+ 120us) server_negotiation.cc:206] Beginning negotiation
0809 15:38:11.809146 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 15:38:11.825872 (+ 16726us) server_negotiation.cc:407] Connection header received
0809 15:38:11.826082 (+ 210us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 15:38:11.826086 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 15:38:11.826151 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 15:38:11.826202 (+ 51us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 15:38:11.826605 (+ 403us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:38:11.827344 (+ 739us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:38:11.828999 (+ 1655us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:38:11.829573 (+ 574us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:38:11.829674 (+ 101us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:38:11.829730 (+ 56us) server_negotiation.cc:1035] Waiting for connection context
0809 15:38:11.830096 (+ 366us) server_negotiation.cc:299] Negotiation successful
0809 15:38:11.830186 (+ 90us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":34}
I20260809 15:38:11.830881 17693 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:38:11.808439 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.89.195:42979 (local address 127.3.89.193:40477)
0809 15:38:11.825777 (+ 17338us) negotiation.cc:107] Waiting for socket to connect
0809 15:38:11.825793 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0809 15:38:11.825953 (+ 160us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 15:38:11.826359 (+ 406us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 15:38:11.826361 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 15:38:11.826379 (+ 18us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 15:38:11.826465 (+ 86us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:38:11.826471 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:38:11.827616 (+ 1145us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:38:11.827619 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:38:11.828823 (+ 1204us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:38:11.828834 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:38:11.829830 (+ 996us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:38:11.829834 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:38:11.829933 (+ 99us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:38:11.829959 (+ 26us) client_negotiation.cc:769] Sending connection context
0809 15:38:11.830657 (+ 698us) client_negotiation.cc:240] Negotiation successful
0809 15:38:11.830742 (+ 85us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":17275,"thread_start_us":107,"threads_started":1}
I20260809 15:38:11.831838 17596 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "40303847baa04254bd3a55f2e59a7dde" candidate_uuid: "28b33706caad4d379f73b37cbf0656c8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "216a25fe11b84aed82d3183ea221a328"
I20260809 15:38:11.832088 17596 raft_consensus.cc:3037] T 40303847baa04254bd3a55f2e59a7dde P 216a25fe11b84aed82d3183ea221a328 [term 0 FOLLOWER]: Advancing to term 1
I20260809 15:38:11.833463 17596 raft_consensus.cc:2454] T 40303847baa04254bd3a55f2e59a7dde P 216a25fe11b84aed82d3183ea221a328 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 28b33706caad4d379f73b37cbf0656c8 in term 1.
I20260809 15:38:11.831851 17597 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "40303847baa04254bd3a55f2e59a7dde" candidate_uuid: "28b33706caad4d379f73b37cbf0656c8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "216a25fe11b84aed82d3183ea221a328" is_pre_election: true
I20260809 15:38:11.834470 17597 raft_consensus.cc:2362] T 40303847baa04254bd3a55f2e59a7dde P 216a25fe11b84aed82d3183ea221a328 [term 1 FOLLOWER]: Leader pre-election vote request: Already granted yes vote for candidate 28b33706caad4d379f73b37cbf0656c8 in term 1. Re-sending same reply.
I20260809 15:38:11.885149 17680 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:38:11.879997 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49352 (local address 127.3.89.193:37409)
0809 15:38:11.880132 (+ 135us) server_negotiation.cc:206] Beginning negotiation
0809 15:38:11.880137 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 15:38:11.880152 (+ 15us) server_negotiation.cc:407] Connection header received
0809 15:38:11.880244 (+ 92us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 15:38:11.880247 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 15:38:11.880321 (+ 74us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 15:38:11.880452 (+ 131us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 15:38:11.880865 (+ 413us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:38:11.881663 (+ 798us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:38:11.882943 (+ 1280us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:38:11.883286 (+ 343us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:38:11.883399 (+ 113us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:38:11.883826 (+ 427us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 15:38:11.884390 (+ 564us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 15:38:11.884511 (+ 121us) server_negotiation.cc:1035] Waiting for connection context
0809 15:38:11.884935 (+ 424us) server_negotiation.cc:299] Negotiation successful
0809 15:38:11.885008 (+ 73us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":57}
I20260809 15:38:11.893853 17449 raft_consensus.cc:1261] T 40303847baa04254bd3a55f2e59a7dde P e7f918425ade4bcbb5ff5f6ece608c08 [term 1 FOLLOWER]: Refusing update from remote peer 28b33706caad4d379f73b37cbf0656c8: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 15:38:11.894114 17597 raft_consensus.cc:1261] T 40303847baa04254bd3a55f2e59a7dde P 216a25fe11b84aed82d3183ea221a328 [term 1 FOLLOWER]: Refusing update from remote peer 28b33706caad4d379f73b37cbf0656c8: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 15:38:11.894801 17689 consensus_queue.cc:1035] T 40303847baa04254bd3a55f2e59a7dde P 28b33706caad4d379f73b37cbf0656c8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "216a25fe11b84aed82d3183ea221a328" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 42979 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 15:38:11.895601 17696 consensus_queue.cc:1035] T 40303847baa04254bd3a55f2e59a7dde P 28b33706caad4d379f73b37cbf0656c8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e7f918425ade4bcbb5ff5f6ece608c08" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 39621 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 15:38:11.924502 17699 mvcc.cc:205] Tried to move back new op lower bound from 7316643397189001216 to 7316643396910592000. Current Snapshot: MvccSnapshot[applied={T|T < 7316643397189001216}]
I20260809 15:38:11.932175 17700 mvcc.cc:205] Tried to move back new op lower bound from 7316643397189001216 to 7316643396910592000. Current Snapshot: MvccSnapshot[applied={T|T < 7316643397189001216}]
I20260809 15:38:11.943820 17140 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:53608:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 15:38:11.944020 17140 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:53608:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 15:38:11.946790 17140 catalog_manager.cc:5782] T 00000000000000000000000000000000 P e056d1c2456c4d4287b5ca44155ff2f2: Sending DeleteTablet for 3 replicas of tablet 40303847baa04254bd3a55f2e59a7dde
I20260809 15:38:11.948017 17429 tablet_service.cc:1514] Processing DeleteTablet for tablet 40303847baa04254bd3a55f2e59a7dde with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 15:38:11 UTC) from {username='kudu', principal='kudu/127.3.89.254@KRBTEST.COM'} at 127.0.0.1:52996
I20260809 15:38:11.948894 17576 tablet_service.cc:1514] Processing DeleteTablet for tablet 40303847baa04254bd3a55f2e59a7dde with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 15:38:11 UTC) from {username='kudu', principal='kudu/127.3.89.254@KRBTEST.COM'} at 127.0.0.1:40190
I20260809 15:38:11.949790 3431 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskm314YR/build/debug/bin/kudu with pid 17185
I20260809 15:38:11.957816 17709 tablet_replica.cc:330] T 40303847baa04254bd3a55f2e59a7dde P 216a25fe11b84aed82d3183ea221a328: stopping tablet replica
I20260809 15:38:11.958228 17709 raft_consensus.cc:2229] T 40303847baa04254bd3a55f2e59a7dde P 216a25fe11b84aed82d3183ea221a328 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 15:38:11.958643 17709 raft_consensus.cc:2258] T 40303847baa04254bd3a55f2e59a7dde P 216a25fe11b84aed82d3183ea221a328 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 15:38:11.959148 17708 tablet_replica.cc:330] T 40303847baa04254bd3a55f2e59a7dde P e7f918425ade4bcbb5ff5f6ece608c08: stopping tablet replica
I20260809 15:38:11.959434 17708 raft_consensus.cc:2229] T 40303847baa04254bd3a55f2e59a7dde P e7f918425ade4bcbb5ff5f6ece608c08 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 15:38:11.959689 17708 raft_consensus.cc:2258] T 40303847baa04254bd3a55f2e59a7dde P e7f918425ade4bcbb5ff5f6ece608c08 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 15:38:11.961046 17708 ts_tablet_manager.cc:1905] T 40303847baa04254bd3a55f2e59a7dde P e7f918425ade4bcbb5ff5f6ece608c08: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 15:38:11.961648 17709 ts_tablet_manager.cc:1905] T 40303847baa04254bd3a55f2e59a7dde P 216a25fe11b84aed82d3183ea221a328: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 15:38:11.963757 17708 ts_tablet_manager.cc:1918] T 40303847baa04254bd3a55f2e59a7dde P e7f918425ade4bcbb5ff5f6ece608c08: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 15:38:11.963869 17708 log.cc:1196] T 40303847baa04254bd3a55f2e59a7dde P e7f918425ade4bcbb5ff5f6ece608c08: Deleting WAL directory at /tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786289732823074-3431-0/minicluster-data/ts-1/wal/wals/40303847baa04254bd3a55f2e59a7dde
I20260809 15:38:11.964192 17708 ts_tablet_manager.cc:1939] T 40303847baa04254bd3a55f2e59a7dde P e7f918425ade4bcbb5ff5f6ece608c08: Deleting consensus metadata
I20260809 15:38:11.965444 17122 catalog_manager.cc:4841] TS e7f918425ade4bcbb5ff5f6ece608c08 (127.3.89.194:39621): tablet 40303847baa04254bd3a55f2e59a7dde (table test-table [id=6600305929fa49dbb0c7f7bea58ccea8]) successfully deleted
W20260809 15:38:11.966238 17126 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.3.89.193:37409 (error 108)
I20260809 15:38:11.966861 17709 ts_tablet_manager.cc:1918] T 40303847baa04254bd3a55f2e59a7dde P 216a25fe11b84aed82d3183ea221a328: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 15:38:11.967139 3431 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskm314YR/build/debug/bin/kudu with pid 17358
I20260809 15:38:11.975163 17709 log.cc:1196] T 40303847baa04254bd3a55f2e59a7dde P 216a25fe11b84aed82d3183ea221a328: Deleting WAL directory at /tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786289732823074-3431-0/minicluster-data/ts-2/wal/wals/40303847baa04254bd3a55f2e59a7dde
I20260809 15:38:11.975602 17709 ts_tablet_manager.cc:1939] T 40303847baa04254bd3a55f2e59a7dde P 216a25fe11b84aed82d3183ea221a328: Deleting consensus metadata
I20260809 15:38:11.977046 17125 catalog_manager.cc:4841] TS 216a25fe11b84aed82d3183ea221a328 (127.3.89.195:42979): tablet 40303847baa04254bd3a55f2e59a7dde (table test-table [id=6600305929fa49dbb0c7f7bea58ccea8]) successfully deleted
I20260809 15:38:11.979808 3431 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskm314YR/build/debug/bin/kudu with pid 17504
I20260809 15:38:11.983907 17678 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:38:11.983543 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.89.193:37409 (local address 127.0.0.1:49362)
0809 15:38:11.983709 (+ 166us) negotiation.cc:107] Waiting for socket to connect
0809 15:38:11.983805 (+ 96us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.89.193:37409: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":44}
W20260809 15:38:11.984130 17126 catalog_manager.cc:4584] TS 28b33706caad4d379f73b37cbf0656c8 (127.3.89.193:37409): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 40303847baa04254bd3a55f2e59a7dde: Network error: Client connection negotiation failed: client connection to 127.3.89.193:37409: connect: Connection refused (error 111)
I20260809 15:38:11.990481 3431 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskm314YR/build/debug/bin/kudu with pid 17105
I20260809 15:38:12.002200 3431 mini_ranger_kms.cc:60] Stopping Ranger KMS...
I20260809 15:38:12.347203 3431 mini_ranger_kms.cc:62] Stopped Ranger KMS
I20260809 15:38:12.347337 3431 mini_ranger.cc:66] Stopping Ranger...
I20260809 15:38:12.692147 3431 mini_ranger.cc:68] Stopped Ranger
2026-08-09 15:38:12.692 UTC [16109] LOG: received smart shutdown request
2026-08-09 15:38:12.696 UTC [16109] LOG: background worker "logical replication launcher" (PID 16118) exited with exit code 1
2026-08-09 15:38:12.696 UTC [16113] LOG: shutting down
2026-08-09 15:38:12.883 UTC [16109] LOG: database system is shut down
2026-08-09T15:38:12Z chronyd exiting
[ OK ] SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers (49161 ms)
[ RUN ] SecurityITest.IPKICACert
Loading random data
Initializing database '/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786289732823074-3431-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 15:38:13 dist-test-slave-2w3w krb5kdc[17720](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 15:38:13 dist-test-slave-2w3w krb5kdc[17720](info): set up 2 sockets
Aug 09 15:38:13 dist-test-slave-2w3w krb5kdc[17720](info): commencing operation
krb5kdc: starting...
W20260809 15:38:14.101274 3431 mini_kdc.cc:121] Time spent starting KDC: real 1.077s user 0.001s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786289732823074-3431-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786289732823074-3431-0/krb5kdc/test-user.keytab.
Aug 09 15:38:14 dist-test-slave-2w3w krb5kdc[17720](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786289894, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T15:38:14Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T15:38:14Z Disabled control of system clock
WARNING: no policy specified for kudu/127.3.89.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.3.89.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.3.89.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786289732823074-3431-0/krb5kdc/kudu_127.3.89.254.keytab.
Entry for principal kudu/127.3.89.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786289732823074-3431-0/krb5kdc/kudu_127.3.89.254.keytab.
I20260809 15:38:14.229957 3431 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskm314YR/build/debug/bin/kudu
/tmp/dist-test-taskm314YR/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786289732823074-3431-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786289732823074-3431-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786289732823074-3431-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786289732823074-3431-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.3.89.254:39401
--webserver_interface=127.3.89.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786289732823074-3431-0/krb5kdc/kudu_127.3.89.254.keytab
--principal=kudu/127.3.89.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.3.89.212:42323
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.3.89.254:39401,127.3.89.253:33211,127.3.89.252:43425
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786289732823074-3431-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786289732823074-3431-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786289732823074-3431-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 15:38:14.338672 17734 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 15:38:14.338912 17734 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 15:38:14.338951 17734 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 15:38:14.342927 17734 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 15:38:14.343055 17734 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 15:38:14.343080 17734 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 15:38:14.343139 17734 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 15:38:14.343171 17734 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 15:38:14.348443 17734 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.3.89.212:42323
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786289732823074-3431-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786289732823074-3431-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.3.89.254:39401,127.3.89.253:33211,127.3.89.252:43425
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786289732823074-3431-0/krb5kdc/kudu_127.3.89.254.keytab
--principal=kudu/127.3.89.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.3.89.254:39401
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786289732823074-3431-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.3.89.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.17734
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786289732823074-3431-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 09 Aug 2026 14:46:55 UTC on ccycloud.quasar-yswkmf.root.comops.site
build id 4576540
I20260809 15:38:14.349431 17734 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 15:38:14.350427 17734 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 15:38:14.359411 17740 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 15:38:14.359402 17739 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 15:38:14.359398 17742 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 15:38:14.359515 17734 server_base.cc:1034] running on GCE node
I20260809 15:38:14.360309 17734 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 15:38:14.361308 17734 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 15:38:14.362500 17734 hybrid_clock.cc:648] HybridClock initialized: now 1786289894362489 us; error 25 us; skew 500 ppm
Aug 09 15:38:14 dist-test-slave-2w3w krb5kdc[17720](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786289894, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.89.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 15:38:14.372128 17734 init.cc:377] Logged in from keytab as kudu/127.3.89.254@KRBTEST.COM (short username kudu)
I20260809 15:38:14.373019 17734 webserver.cc:460] Webserver started at http://127.3.89.254:36285/ using document root <none> and password file <none>
I20260809 15:38:14.373567 17734 fs_manager.cc:359] Metadata directory not provided
I20260809 15:38:14.373662 17734 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 15:38:14.373900 17734 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 15:38:14.376085 17734 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786289732823074-3431-0/minicluster-data/master-0/data/instance:
uuid: "16412a81332747479bfb62d9280280c3"
format_stamp: "Formatted at 2026-08-09 15:38:14 on dist-test-slave-2w3w"
server_key: "9d31f6724a77bc0b40c0db1ef1dd3d10"
server_key_iv: "ab947f63bf48abe0fc811b2f8c7afd6c"
server_key_version: "encryptionkey@0"
I20260809 15:38:14.376648 17734 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786289732823074-3431-0/minicluster-data/master-0/wal/instance:
uuid: "16412a81332747479bfb62d9280280c3"
format_stamp: "Formatted at 2026-08-09 15:38:14 on dist-test-slave-2w3w"
server_key: "9d31f6724a77bc0b40c0db1ef1dd3d10"
server_key_iv: "ab947f63bf48abe0fc811b2f8c7afd6c"
server_key_version: "encryptionkey@0"
I20260809 15:38:14.380189 17734 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.004s sys 0.000s
I20260809 15:38:14.382495 17749 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 15:38:14.383925 17734 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260809 15:38:14.384080 17734 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786289732823074-3431-0/minicluster-data/master-0/data,/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786289732823074-3431-0/minicluster-data/master-0/wal
uuid: "16412a81332747479bfb62d9280280c3"
format_stamp: "Formatted at 2026-08-09 15:38:14 on dist-test-slave-2w3w"
server_key: "9d31f6724a77bc0b40c0db1ef1dd3d10"
server_key_iv: "ab947f63bf48abe0fc811b2f8c7afd6c"
server_key_version: "encryptionkey@0"
I20260809 15:38:14.384193 17734 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786289732823074-3431-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786289732823074-3431-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786289732823074-3431-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 15:38:14.405550 17734 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 15:38:14.408659 17734 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 15:38:14.408871 17734 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 15:38:14.416296 17734 rpc_server.cc:307] RPC server started. Bound to: 127.3.89.254:39401
I20260809 15:38:14.416349 17806 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.3.89.254:39401 every 8 connection(s)
I20260809 15:38:14.417291 17734 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786289732823074-3431-0/minicluster-data/master-0/data/info.pb
I20260809 15:38:14.420127 17807 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 15:38:14.424016 17807 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.3.89.254" port: 39401 } has no permanent_uuid. Determining permanent_uuid...
I20260809 15:38:14.426470 3431 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskm314YR/build/debug/bin/kudu as pid 17734
I20260809 15:38:14.426657 3431 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786289732823074-3431-0/minicluster-data/master-0/wal/instance
I20260809 15:38:14.427095 3431 external_mini_cluster.cc:1428] Setting key b71bdc58605d96216aeaf134dbf7173a
Aug 09 15:38:14 dist-test-slave-2w3w krb5kdc[17720](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786289894, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.89.254@KRBTEST.COM for kudu/127.3.89.254@KRBTEST.COM
WARNING: no policy specified for kudu/127.3.89.253@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.3.89.253@KRBTEST.COM" created.
I20260809 15:38:14.440011 17809 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:38:14.426617 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.89.254:39401 (local address 127.0.0.1:51646)
0809 15:38:14.427203 (+ 586us) negotiation.cc:107] Waiting for socket to connect
0809 15:38:14.427261 (+ 58us) client_negotiation.cc:174] Beginning negotiation
0809 15:38:14.428531 (+ 1270us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 15:38:14.429214 (+ 683us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 15:38:14.429219 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 15:38:14.429657 (+ 438us) client_negotiation.cc:189] Negotiated authn=SASL
0809 15:38:14.429894 (+ 237us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:38:14.429907 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:38:14.431155 (+ 1248us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:38:14.431158 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:38:14.431864 (+ 706us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:38:14.431873 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:38:14.432560 (+ 687us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:38:14.432563 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:38:14.432667 (+ 104us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:38:14.433336 (+ 669us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 15:38:14.433351 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0809 15:38:14.434809 (+ 1458us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 15:38:14.436801 (+ 1992us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 15:38:14.436811 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 15:38:14.436820 (+ 9us) client_negotiation.cc:760] calling sasl_client_step()
0809 15:38:14.437128 (+ 308us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 15:38:14.437463 (+ 335us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 15:38:14.437467 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 15:38:14.437471 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0809 15:38:14.437594 (+ 123us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 15:38:14.438196 (+ 602us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 15:38:14.438202 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 15:38:14.438346 (+ 144us) client_negotiation.cc:769] Sending connection context
0809 15:38:14.438538 (+ 192us) client_negotiation.cc:240] Negotiation successful
0809 15:38:14.439022 (+ 484us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":360,"mutex_wait_us":179,"thread_start_us":148,"threads_started":1}
I20260809 15:38:14.440021 17810 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:38:14.426617 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51646 (local address 127.3.89.254:39401)
0809 15:38:14.427194 (+ 577us) server_negotiation.cc:206] Beginning negotiation
0809 15:38:14.427212 (+ 18us) server_negotiation.cc:399] Waiting for connection header
0809 15:38:14.427398 (+ 186us) server_negotiation.cc:407] Connection header received
0809 15:38:14.428836 (+ 1438us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 15:38:14.428840 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 15:38:14.429009 (+ 169us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 15:38:14.429216 (+ 207us) server_negotiation.cc:226] Negotiated authn=SASL
0809 15:38:14.430063 (+ 847us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:38:14.430962 (+ 899us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:38:14.432038 (+ 1076us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:38:14.432394 (+ 356us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:38:14.432582 (+ 188us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:38:14.435002 (+ 2420us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 15:38:14.435010 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 15:38:14.435028 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 15:38:14.435050 (+ 22us) server_negotiation.cc:922] calling sasl_server_start()
0809 15:38:14.436680 (+ 1630us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 15:38:14.437246 (+ 566us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 15:38:14.437254 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 15:38:14.437255 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 15:38:14.437361 (+ 106us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 15:38:14.437702 (+ 341us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 15:38:14.437705 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 15:38:14.437707 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 15:38:14.438067 (+ 360us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 15:38:14.438216 (+ 149us) server_negotiation.cc:1035] Waiting for connection context
0809 15:38:14.438599 (+ 383us) server_negotiation.cc:299] Negotiation successful
0809 15:38:14.438944 (+ 345us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":24,"server-negotiator.queue_time_us":455,"spinlock_wait_cycles":48896,"thread_start_us":161,"threads_started":1}
I20260809 15:38:14.442530 17807 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.3.89.253" port: 33211 } has no permanent_uuid. Determining permanent_uuid...
I20260809 15:38:14.443369 17809 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:38:14.443102 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.89.253:33211 (local address 127.0.0.1:39772)
0809 15:38:14.443225 (+ 123us) negotiation.cc:107] Waiting for socket to connect
0809 15:38:14.443306 (+ 81us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.89.253:33211: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":49}
W20260809 15:38:14.443627 17750 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.3.89.253:33211: connect: Connection refused (error 111)
I20260809 15:38:14.444479 17809 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:38:14.444262 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.89.253:33211 (local address 127.0.0.1:39786)
0809 15:38:14.444372 (+ 110us) negotiation.cc:107] Waiting for socket to connect
0809 15:38:14.444416 (+ 44us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.89.253:33211: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":48}
W20260809 15:38:14.444650 17807 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.3.89.253:33211: Network error: Client connection negotiation failed: client connection to 127.3.89.253:33211: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.3.89.253 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786289732823074-3431-0/krb5kdc/kudu_127.3.89.253.keytab.
Entry for principal kudu/127.3.89.253 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786289732823074-3431-0/krb5kdc/kudu_127.3.89.253.keytab.
I20260809 15:38:14.456326 3431 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskm314YR/build/debug/bin/kudu
/tmp/dist-test-taskm314YR/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786289732823074-3431-0/minicluster-data/master-1/wal
--fs_data_dirs=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786289732823074-3431-0/minicluster-data/master-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786289732823074-3431-0/minicluster-data/master-1/logs
--server_dump_info_path=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786289732823074-3431-0/minicluster-data/master-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.3.89.253:33211
--webserver_interface=127.3.89.253
--webserver_port=0
--keytab_file=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786289732823074-3431-0/krb5kdc/kudu_127.3.89.253.keytab
--principal=kudu/127.3.89.253
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.3.89.212:42323
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.3.89.254:39401,127.3.89.253:33211,127.3.89.252:43425
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786289732823074-3431-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786289732823074-3431-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786289732823074-3431-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260809 15:38:14.493858 17807 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.3.89.253" port: 33211 } attempt: 1
I20260809 15:38:14.494798 17809 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:38:14.494476 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.89.253:33211 (local address 127.0.0.1:39794)
0809 15:38:14.494639 (+ 163us) negotiation.cc:107] Waiting for socket to connect
0809 15:38:14.494702 (+ 63us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.89.253:33211: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":69}
I20260809 15:38:14.495565 17809 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:38:14.495355 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.89.253:33211 (local address 127.0.0.1:39806)
0809 15:38:14.495460 (+ 105us) negotiation.cc:107] Waiting for socket to connect
0809 15:38:14.495508 (+ 48us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.89.253:33211: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":46}
W20260809 15:38:14.495723 17807 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.3.89.253:33211: Network error: Client connection negotiation failed: client connection to 127.3.89.253:33211: connect: Connection refused (error 111)
I20260809 15:38:14.563903 17807 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.3.89.253" port: 33211 } attempt: 2
I20260809 15:38:14.564762 17809 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:38:14.564468 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.89.253:33211 (local address 127.0.0.1:39808)
0809 15:38:14.564623 (+ 155us) negotiation.cc:107] Waiting for socket to connect
0809 15:38:14.564685 (+ 62us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.89.253:33211: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":60}
I20260809 15:38:14.565487 17809 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:38:14.565294 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.89.253:33211 (local address 127.0.0.1:39824)
0809 15:38:14.565389 (+ 95us) negotiation.cc:107] Waiting for socket to connect
0809 15:38:14.565431 (+ 42us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.89.253:33211: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":40}
W20260809 15:38:14.565635 17807 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.3.89.253:33211: Network error: Client connection negotiation failed: client connection to 127.3.89.253:33211: connect: Connection refused (error 111)
W20260809 15:38:14.566838 17814 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 15:38:14.567124 17814 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 15:38:14.567219 17814 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 15:38:14.571178 17814 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 15:38:14.571225 17814 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 15:38:14.571275 17814 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 15:38:14.571292 17814 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 15:38:14.571334 17814 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 15:38:14.576628 17814 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.3.89.212:42323
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786289732823074-3431-0/minicluster-data/master-1/data
--fs_wal_dir=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786289732823074-3431-0/minicluster-data/master-1/wal
--ipki_ca_key_size=768
--master_addresses=127.3.89.254:39401,127.3.89.253:33211,127.3.89.252:43425
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786289732823074-3431-0/krb5kdc/kudu_127.3.89.253.keytab
--principal=kudu/127.3.89.253
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.3.89.253:33211
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786289732823074-3431-0/minicluster-data/master-1/data/info.pb
--webserver_interface=127.3.89.253
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.17814
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786289732823074-3431-0/minicluster-data/master-1/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 09 Aug 2026 14:46:55 UTC on ccycloud.quasar-yswkmf.root.comops.site
build id 4576540
I20260809 15:38:14.577598 17814 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 15:38:14.578603 17814 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 15:38:14.587041 17822 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 15:38:14.587041 17820 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 15:38:14.587045 17819 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 15:38:14.587639 17814 server_base.cc:1034] running on GCE node
I20260809 15:38:14.588259 17814 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 15:38:14.589360 17814 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 15:38:14.590570 17814 hybrid_clock.cc:648] HybridClock initialized: now 1786289894590543 us; error 46 us; skew 500 ppm
Aug 09 15:38:14 dist-test-slave-2w3w krb5kdc[17720](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786289894, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.89.253@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 15:38:14.593556 17814 init.cc:377] Logged in from keytab as kudu/127.3.89.253@KRBTEST.COM (short username kudu)
I20260809 15:38:14.594521 17814 webserver.cc:460] Webserver started at http://127.3.89.253:39987/ using document root <none> and password file <none>
I20260809 15:38:14.595184 17814 fs_manager.cc:359] Metadata directory not provided
I20260809 15:38:14.595283 17814 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 15:38:14.595530 17814 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 15:38:14.597579 17814 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786289732823074-3431-0/minicluster-data/master-1/data/instance:
uuid: "78e16fb3cd5f4817a798aaf26d28231f"
format_stamp: "Formatted at 2026-08-09 15:38:14 on dist-test-slave-2w3w"
server_key: "21082a52af70a062311b572b7c2b4a21"
server_key_iv: "e4aa09701172ee26d8ba19080fde5377"
server_key_version: "encryptionkey@0"
I20260809 15:38:14.598254 17814 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786289732823074-3431-0/minicluster-data/master-1/wal/instance:
uuid: "78e16fb3cd5f4817a798aaf26d28231f"
format_stamp: "Formatted at 2026-08-09 15:38:14 on dist-test-slave-2w3w"
server_key: "21082a52af70a062311b572b7c2b4a21"
server_key_iv: "e4aa09701172ee26d8ba19080fde5377"
server_key_version: "encryptionkey@0"
I20260809 15:38:14.601939 17814 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.006s sys 0.000s
I20260809 15:38:14.604446 17829 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 15:38:14.605738 17814 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260809 15:38:14.605914 17814 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786289732823074-3431-0/minicluster-data/master-1/data,/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786289732823074-3431-0/minicluster-data/master-1/wal
uuid: "78e16fb3cd5f4817a798aaf26d28231f"
format_stamp: "Formatted at 2026-08-09 15:38:14 on dist-test-slave-2w3w"
server_key: "21082a52af70a062311b572b7c2b4a21"
server_key_iv: "e4aa09701172ee26d8ba19080fde5377"
server_key_version: "encryptionkey@0"
I20260809 15:38:14.606040 17814 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786289732823074-3431-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786289732823074-3431-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786289732823074-3431-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 15:38:14.622715 17814 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 15:38:14.626094 17814 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 15:38:14.626329 17814 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 15:38:14.633617 17814 rpc_server.cc:307] RPC server started. Bound to: 127.3.89.253:33211
I20260809 15:38:14.633698 17883 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.3.89.253:33211 every 8 connection(s)
I20260809 15:38:14.634596 17814 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786289732823074-3431-0/minicluster-data/master-1/data/info.pb
I20260809 15:38:14.637262 17884 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 15:38:14.640877 17884 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.3.89.254" port: 39401 } has no permanent_uuid. Determining permanent_uuid...
I20260809 15:38:14.642980 3431 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskm314YR/build/debug/bin/kudu as pid 17814
I20260809 15:38:14.643167 3431 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786289732823074-3431-0/minicluster-data/master-1/wal/instance
I20260809 15:38:14.643442 3431 external_mini_cluster.cc:1428] Setting key 0b220078855a8a481b317d015601600b
Aug 09 15:38:14 dist-test-slave-2w3w krb5kdc[17720](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786289894, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.89.253@KRBTEST.COM for kudu/127.3.89.254@KRBTEST.COM
WARNING: no policy specified for kudu/127.3.89.252@KRBTEST.COM; defaulting to no policy
I20260809 15:38:14.655242 17810 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:38:14.642625 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51662 (local address 127.3.89.254:39401)
0809 15:38:14.642851 (+ 226us) server_negotiation.cc:206] Beginning negotiation
0809 15:38:14.642857 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 15:38:14.643686 (+ 829us) server_negotiation.cc:407] Connection header received
0809 15:38:14.644581 (+ 895us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 15:38:14.644587 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 15:38:14.644653 (+ 66us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 15:38:14.644762 (+ 109us) server_negotiation.cc:226] Negotiated authn=SASL
0809 15:38:14.646100 (+ 1338us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:38:14.646630 (+ 530us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:38:14.648449 (+ 1819us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:38:14.648710 (+ 261us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:38:14.648795 (+ 85us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:38:14.651403 (+ 2608us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 15:38:14.651406 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 15:38:14.651409 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 15:38:14.651435 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0809 15:38:14.652997 (+ 1562us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 15:38:14.653513 (+ 516us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 15:38:14.653517 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 15:38:14.653519 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 15:38:14.653609 (+ 90us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 15:38:14.654019 (+ 410us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 15:38:14.654022 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 15:38:14.654023 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 15:38:14.654196 (+ 173us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 15:38:14.654307 (+ 111us) server_negotiation.cc:1035] Waiting for connection context
0809 15:38:14.654882 (+ 575us) server_negotiation.cc:299] Negotiation successful
0809 15:38:14.655068 (+ 186us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":110}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.3.89.252@KRBTEST.COM" created.
I20260809 15:38:14.656044 17886 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:38:14.643004 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.89.254:39401 (local address 127.0.0.1:51662)
0809 15:38:14.643524 (+ 520us) negotiation.cc:107] Waiting for socket to connect
0809 15:38:14.643574 (+ 50us) client_negotiation.cc:174] Beginning negotiation
0809 15:38:14.644378 (+ 804us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 15:38:14.644903 (+ 525us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 15:38:14.644909 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 15:38:14.645218 (+ 309us) client_negotiation.cc:189] Negotiated authn=SASL
0809 15:38:14.645930 (+ 712us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:38:14.645952 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:38:14.646781 (+ 829us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:38:14.646786 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:38:14.648294 (+ 1508us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:38:14.648310 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:38:14.648879 (+ 569us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:38:14.648883 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:38:14.649081 (+ 198us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:38:14.649726 (+ 645us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 15:38:14.649741 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0809 15:38:14.651231 (+ 1490us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 15:38:14.653118 (+ 1887us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 15:38:14.653124 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 15:38:14.653136 (+ 12us) client_negotiation.cc:760] calling sasl_client_step()
0809 15:38:14.653396 (+ 260us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 15:38:14.653723 (+ 327us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 15:38:14.653728 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 15:38:14.653729 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0809 15:38:14.653910 (+ 181us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 15:38:14.654315 (+ 405us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 15:38:14.654328 (+ 13us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 15:38:14.654572 (+ 244us) client_negotiation.cc:769] Sending connection context
0809 15:38:14.654770 (+ 198us) client_negotiation.cc:240] Negotiation successful
0809 15:38:14.655088 (+ 318us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":322,"thread_start_us":166,"threads_started":1}
I20260809 15:38:14.656816 17807 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.3.89.253" port: 33211 } attempt: 3
I20260809 15:38:14.657624 17884 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.3.89.253" port: 33211 } has no permanent_uuid. Determining permanent_uuid...
Aug 09 15:38:14 dist-test-slave-2w3w krb5kdc[17720](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786289894, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.89.253@KRBTEST.COM for kudu/127.3.89.253@KRBTEST.COM
Aug 09 15:38:14 dist-test-slave-2w3w krb5kdc[17720](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786289894, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.89.254@KRBTEST.COM for kudu/127.3.89.253@KRBTEST.COM
I20260809 15:38:14.668354 17886 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:38:14.658148 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.89.253:33211 (local address 127.0.0.1:39848)
0809 15:38:14.658306 (+ 158us) negotiation.cc:107] Waiting for socket to connect
0809 15:38:14.658321 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0809 15:38:14.658385 (+ 64us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 15:38:14.658873 (+ 488us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 15:38:14.658877 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 15:38:14.659079 (+ 202us) client_negotiation.cc:189] Negotiated authn=SASL
0809 15:38:14.659290 (+ 211us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:38:14.659299 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:38:14.660511 (+ 1212us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:38:14.660515 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:38:14.661306 (+ 791us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:38:14.661313 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:38:14.661858 (+ 545us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:38:14.661861 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:38:14.661950 (+ 89us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:38:14.662550 (+ 600us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 15:38:14.662562 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0809 15:38:14.664208 (+ 1646us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 15:38:14.666332 (+ 2124us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 15:38:14.666335 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 15:38:14.666336 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0809 15:38:14.666733 (+ 397us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 15:38:14.667144 (+ 411us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 15:38:14.667151 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 15:38:14.667154 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0809 15:38:14.667255 (+ 101us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 15:38:14.667801 (+ 546us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 15:38:14.667804 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 15:38:14.667895 (+ 91us) client_negotiation.cc:769] Sending connection context
0809 15:38:14.668025 (+ 130us) client_negotiation.cc:240] Negotiation successful
0809 15:38:14.668155 (+ 130us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":94,"mutex_wait_us":76}
I20260809 15:38:14.668612 17890 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:38:14.658212 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39848 (local address 127.3.89.253:33211)
0809 15:38:14.658604 (+ 392us) server_negotiation.cc:206] Beginning negotiation
0809 15:38:14.658615 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0809 15:38:14.658631 (+ 16us) server_negotiation.cc:407] Connection header received
0809 15:38:14.658671 (+ 40us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 15:38:14.658674 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 15:38:14.658717 (+ 43us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 15:38:14.658857 (+ 140us) server_negotiation.cc:226] Negotiated authn=SASL
0809 15:38:14.659424 (+ 567us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:38:14.660375 (+ 951us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:38:14.661437 (+ 1062us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:38:14.661664 (+ 227us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:38:14.661858 (+ 194us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:38:14.664349 (+ 2491us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 15:38:14.664359 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 15:38:14.664375 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 15:38:14.664395 (+ 20us) server_negotiation.cc:922] calling sasl_server_start()
0809 15:38:14.666141 (+ 1746us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 15:38:14.666854 (+ 713us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 15:38:14.666859 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 15:38:14.666860 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 15:38:14.666939 (+ 79us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 15:38:14.667358 (+ 419us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 15:38:14.667360 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 15:38:14.667361 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 15:38:14.667638 (+ 277us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 15:38:14.667838 (+ 200us) server_negotiation.cc:1035] Waiting for connection context
0809 15:38:14.668114 (+ 276us) server_negotiation.cc:299] Negotiation successful
0809 15:38:14.668299 (+ 185us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":319,"thread_start_us":156,"threads_started":1}
I20260809 15:38:14.669351 17809 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:38:14.657206 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.89.253:33211 (local address 127.0.0.1:39840)
0809 15:38:14.657354 (+ 148us) negotiation.cc:107] Waiting for socket to connect
0809 15:38:14.657365 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0809 15:38:14.657449 (+ 84us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 15:38:14.658269 (+ 820us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 15:38:14.658272 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 15:38:14.658463 (+ 191us) client_negotiation.cc:189] Negotiated authn=SASL
0809 15:38:14.659029 (+ 566us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:38:14.659040 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:38:14.660233 (+ 1193us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:38:14.660236 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:38:14.661072 (+ 836us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:38:14.661082 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:38:14.661761 (+ 679us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:38:14.661764 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:38:14.661889 (+ 125us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:38:14.662353 (+ 464us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 15:38:14.662374 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0809 15:38:14.664334 (+ 1960us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 15:38:14.667418 (+ 3084us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 15:38:14.667422 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 15:38:14.667424 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 15:38:14.667621 (+ 197us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 15:38:14.667930 (+ 309us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 15:38:14.667934 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 15:38:14.667936 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 15:38:14.668026 (+ 90us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 15:38:14.668816 (+ 790us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 15:38:14.668820 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 15:38:14.668927 (+ 107us) client_negotiation.cc:769] Sending connection context
0809 15:38:14.669040 (+ 113us) client_negotiation.cc:240] Negotiation successful
0809 15:38:14.669196 (+ 156us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":96}
I20260809 15:38:14.669426 17889 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:38:14.657554 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39840 (local address 127.3.89.253:33211)
0809 15:38:14.657844 (+ 290us) server_negotiation.cc:206] Beginning negotiation
0809 15:38:14.657854 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0809 15:38:14.657871 (+ 17us) server_negotiation.cc:407] Connection header received
0809 15:38:14.657904 (+ 33us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 15:38:14.657907 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 15:38:14.658115 (+ 208us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 15:38:14.659146 (+ 1031us) server_negotiation.cc:226] Negotiated authn=SASL
0809 15:38:14.659307 (+ 161us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:38:14.660062 (+ 755us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:38:14.661217 (+ 1155us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:38:14.661632 (+ 415us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:38:14.661759 (+ 127us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:38:14.664540 (+ 2781us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 15:38:14.664546 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 15:38:14.664551 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 15:38:14.664581 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0809 15:38:14.667292 (+ 2711us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 15:38:14.667735 (+ 443us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 15:38:14.667739 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 15:38:14.667741 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 15:38:14.667826 (+ 85us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 15:38:14.668130 (+ 304us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 15:38:14.668135 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 15:38:14.668137 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 15:38:14.668368 (+ 231us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 15:38:14.668815 (+ 447us) server_negotiation.cc:1035] Waiting for connection context
0809 15:38:14.669158 (+ 343us) server_negotiation.cc:299] Negotiation successful
0809 15:38:14.669268 (+ 110us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":4,"server-negotiator.queue_time_us":195,"thread_start_us":115,"threads_started":1}
I20260809 15:38:14.670771 17807 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.3.89.252" port: 43425 } has no permanent_uuid. Determining permanent_uuid...
I20260809 15:38:14.671005 17884 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.3.89.252" port: 43425 } has no permanent_uuid. Determining permanent_uuid...
I20260809 15:38:14.671619 17809 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:38:14.671306 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.89.252:43425 (local address 127.0.0.1:53352)
0809 15:38:14.671502 (+ 196us) negotiation.cc:107] Waiting for socket to connect
0809 15:38:14.671554 (+ 52us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.89.252:43425: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":129}
I20260809 15:38:14.671670 17886 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:38:14.671441 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.89.252:43425 (local address 127.0.0.1:53366)
0809 15:38:14.671536 (+ 95us) negotiation.cc:107] Waiting for socket to connect
0809 15:38:14.671603 (+ 67us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.89.252:43425: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":29}
W20260809 15:38:14.671868 17830 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.3.89.252:43425: connect: Connection refused (error 111)
I20260809 15:38:14.672353 17809 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:38:14.672114 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.89.252:43425 (local address 127.0.0.1:53370)
0809 15:38:14.672250 (+ 136us) negotiation.cc:107] Waiting for socket to connect
0809 15:38:14.672307 (+ 57us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.89.252:43425: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":75}
W20260809 15:38:14.672540 17807 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.3.89.252:43425: Network error: Client connection negotiation failed: client connection to 127.3.89.252:43425: connect: Connection refused (error 111)
I20260809 15:38:14.672784 17886 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:38:14.672602 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.89.252:43425 (local address 127.0.0.1:53384)
0809 15:38:14.672694 (+ 92us) negotiation.cc:107] Waiting for socket to connect
0809 15:38:14.672736 (+ 42us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.89.252:43425: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":26}
W20260809 15:38:14.673058 17884 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.3.89.252:43425: Network error: Client connection negotiation failed: client connection to 127.3.89.252:43425: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.3.89.252 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786289732823074-3431-0/krb5kdc/kudu_127.3.89.252.keytab.
Entry for principal kudu/127.3.89.252 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786289732823074-3431-0/krb5kdc/kudu_127.3.89.252.keytab.
I20260809 15:38:14.674817 3431 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskm314YR/build/debug/bin/kudu
/tmp/dist-test-taskm314YR/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786289732823074-3431-0/minicluster-data/master-2/wal
--fs_data_dirs=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786289732823074-3431-0/minicluster-data/master-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786289732823074-3431-0/minicluster-data/master-2/logs
--server_dump_info_path=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786289732823074-3431-0/minicluster-data/master-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.3.89.252:43425
--webserver_interface=127.3.89.252
--webserver_port=0
--keytab_file=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786289732823074-3431-0/krb5kdc/kudu_127.3.89.252.keytab
--principal=kudu/127.3.89.252
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.3.89.212:42323
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.3.89.254:39401,127.3.89.253:33211,127.3.89.252:43425
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786289732823074-3431-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786289732823074-3431-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786289732823074-3431-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260809 15:38:14.703707 17807 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.3.89.252" port: 43425 } attempt: 1
I20260809 15:38:14.704595 17809 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:38:14.704261 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.89.252:43425 (local address 127.0.0.1:53386)
0809 15:38:14.704456 (+ 195us) negotiation.cc:107] Waiting for socket to connect
0809 15:38:14.704516 (+ 60us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.89.252:43425: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":55}
I20260809 15:38:14.705440 17809 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:38:14.705172 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.89.252:43425 (local address 127.0.0.1:53396)
0809 15:38:14.705331 (+ 159us) negotiation.cc:107] Waiting for socket to connect
0809 15:38:14.705380 (+ 49us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.89.252:43425: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":72}
W20260809 15:38:14.705636 17807 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.3.89.252:43425: Network error: Client connection negotiation failed: client connection to 127.3.89.252:43425: connect: Connection refused (error 111)
I20260809 15:38:14.722292 17884 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.3.89.252" port: 43425 } attempt: 1
I20260809 15:38:14.723193 17886 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:38:14.722853 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.89.252:43425 (local address 127.0.0.1:53398)
0809 15:38:14.723032 (+ 179us) negotiation.cc:107] Waiting for socket to connect
0809 15:38:14.723107 (+ 75us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.89.252:43425: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":64}
I20260809 15:38:14.724048 17886 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:38:14.723805 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.89.252:43425 (local address 127.0.0.1:53402)
0809 15:38:14.723924 (+ 119us) negotiation.cc:107] Waiting for socket to connect
0809 15:38:14.723980 (+ 56us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.89.252:43425: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":45}
W20260809 15:38:14.724241 17884 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.3.89.252:43425: Network error: Client connection negotiation failed: client connection to 127.3.89.252:43425: connect: Connection refused (error 111)
I20260809 15:38:14.780927 17807 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.3.89.252" port: 43425 } attempt: 2
I20260809 15:38:14.781747 17809 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:38:14.781466 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.89.252:43425 (local address 127.0.0.1:53408)
0809 15:38:14.781612 (+ 146us) negotiation.cc:107] Waiting for socket to connect
0809 15:38:14.781670 (+ 58us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.89.252:43425: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":56}
I20260809 15:38:14.782464 17809 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:38:14.782241 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.89.252:43425 (local address 127.0.0.1:53420)
0809 15:38:14.782363 (+ 122us) negotiation.cc:107] Waiting for socket to connect
0809 15:38:14.782412 (+ 49us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.89.252:43425: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":60}
W20260809 15:38:14.782590 17807 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.3.89.252:43425: Network error: Client connection negotiation failed: client connection to 127.3.89.252:43425: connect: Connection refused (error 111)
W20260809 15:38:14.783893 17892 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 15:38:14.784164 17892 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 15:38:14.784261 17892 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 15:38:14.788297 17892 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 15:38:14.788375 17892 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 15:38:14.788414 17892 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 15:38:14.788455 17892 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 15:38:14.788496 17892 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 15:38:14.792420 17884 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.3.89.252" port: 43425 } attempt: 2
I20260809 15:38:14.793177 17886 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:38:14.792897 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.89.252:43425 (local address 127.0.0.1:53422)
0809 15:38:14.793040 (+ 143us) negotiation.cc:107] Waiting for socket to connect
0809 15:38:14.793101 (+ 61us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.89.252:43425: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":54}
I20260809 15:38:14.793934 17886 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:38:14.793680 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.89.252:43425 (local address 127.0.0.1:53430)
0809 15:38:14.793787 (+ 107us) negotiation.cc:107] Waiting for socket to connect
0809 15:38:14.793852 (+ 65us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.89.252:43425: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":42}
W20260809 15:38:14.794070 17884 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.3.89.252:43425: Network error: Client connection negotiation failed: client connection to 127.3.89.252:43425: connect: Connection refused (error 111)
I20260809 15:38:14.794145 17892 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.3.89.212:42323
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786289732823074-3431-0/minicluster-data/master-2/data
--fs_wal_dir=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786289732823074-3431-0/minicluster-data/master-2/wal
--ipki_ca_key_size=768
--master_addresses=127.3.89.254:39401,127.3.89.253:33211,127.3.89.252:43425
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786289732823074-3431-0/krb5kdc/kudu_127.3.89.252.keytab
--principal=kudu/127.3.89.252
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.3.89.252:43425
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786289732823074-3431-0/minicluster-data/master-2/data/info.pb
--webserver_interface=127.3.89.252
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.17892
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786289732823074-3431-0/minicluster-data/master-2/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 09 Aug 2026 14:46:55 UTC on ccycloud.quasar-yswkmf.root.comops.site
build id 4576540
I20260809 15:38:14.795164 17892 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 15:38:14.796073 17892 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 15:38:14.804104 17900 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 15:38:14.804121 17897 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 15:38:14.804144 17898 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 15:38:14.804471 17892 server_base.cc:1034] running on GCE node
I20260809 15:38:14.805068 17892 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 15:38:14.806090 17892 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 15:38:14.807286 17892 hybrid_clock.cc:648] HybridClock initialized: now 1786289894807271 us; error 29 us; skew 500 ppm
Aug 09 15:38:14 dist-test-slave-2w3w krb5kdc[17720](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786289894, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.89.252@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 15:38:14.810333 17892 init.cc:377] Logged in from keytab as kudu/127.3.89.252@KRBTEST.COM (short username kudu)
I20260809 15:38:14.811183 17892 webserver.cc:460] Webserver started at http://127.3.89.252:39509/ using document root <none> and password file <none>
I20260809 15:38:14.811667 17892 fs_manager.cc:359] Metadata directory not provided
I20260809 15:38:14.811723 17892 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 15:38:14.811884 17892 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 15:38:14.813751 17892 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786289732823074-3431-0/minicluster-data/master-2/data/instance:
uuid: "c835f885d12d4b1fbcf32bf63fa99c30"
format_stamp: "Formatted at 2026-08-09 15:38:14 on dist-test-slave-2w3w"
server_key: "f05b6ee7bab66e6a76e881c3b6db5a73"
server_key_iv: "cb5fce0b8a5ac618a38077fd262d3780"
server_key_version: "encryptionkey@0"
I20260809 15:38:14.814296 17892 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786289732823074-3431-0/minicluster-data/master-2/wal/instance:
uuid: "c835f885d12d4b1fbcf32bf63fa99c30"
format_stamp: "Formatted at 2026-08-09 15:38:14 on dist-test-slave-2w3w"
server_key: "f05b6ee7bab66e6a76e881c3b6db5a73"
server_key_iv: "cb5fce0b8a5ac618a38077fd262d3780"
server_key_version: "encryptionkey@0"
I20260809 15:38:14.818467 17892 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.003s
I20260809 15:38:14.820856 17907 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 15:38:14.822052 17892 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260809 15:38:14.822147 17892 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786289732823074-3431-0/minicluster-data/master-2/data,/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786289732823074-3431-0/minicluster-data/master-2/wal
uuid: "c835f885d12d4b1fbcf32bf63fa99c30"
format_stamp: "Formatted at 2026-08-09 15:38:14 on dist-test-slave-2w3w"
server_key: "f05b6ee7bab66e6a76e881c3b6db5a73"
server_key_iv: "cb5fce0b8a5ac618a38077fd262d3780"
server_key_version: "encryptionkey@0"
I20260809 15:38:14.822232 17892 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786289732823074-3431-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786289732823074-3431-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786289732823074-3431-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 15:38:14.842866 17892 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 15:38:14.846058 17892 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 15:38:14.846195 17892 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 15:38:14.853248 17892 rpc_server.cc:307] RPC server started. Bound to: 127.3.89.252:43425
I20260809 15:38:14.853365 17963 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.3.89.252:43425 every 8 connection(s)
I20260809 15:38:14.854137 17892 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786289732823074-3431-0/minicluster-data/master-2/data/info.pb
I20260809 15:38:14.856710 17964 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 15:38:14.860404 17964 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.3.89.254" port: 39401 } has no permanent_uuid. Determining permanent_uuid...
I20260809 15:38:14.861052 3431 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskm314YR/build/debug/bin/kudu as pid 17892
I20260809 15:38:14.861183 3431 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786289732823074-3431-0/minicluster-data/master-2/wal/instance
I20260809 15:38:14.861447 3431 external_mini_cluster.cc:1428] Setting key da7144cd909c44405cc2abe99cf17059
Aug 09 15:38:14 dist-test-slave-2w3w krb5kdc[17720](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786289894, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.89.252@KRBTEST.COM for kudu/127.3.89.254@KRBTEST.COM
Aug 09 15:38:14 dist-test-slave-2w3w krb5kdc[17720](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786289894, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.3.89.252@KRBTEST.COM
I20260809 15:38:14.874709 17810 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:38:14.862324 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51664 (local address 127.3.89.254:39401)
0809 15:38:14.862445 (+ 121us) server_negotiation.cc:206] Beginning negotiation
0809 15:38:14.862449 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 15:38:14.863248 (+ 799us) server_negotiation.cc:407] Connection header received
0809 15:38:14.864034 (+ 786us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 15:38:14.864037 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 15:38:14.864084 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 15:38:14.864171 (+ 87us) server_negotiation.cc:226] Negotiated authn=SASL
0809 15:38:14.865457 (+ 1286us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:38:14.866182 (+ 725us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:38:14.867569 (+ 1387us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:38:14.867896 (+ 327us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:38:14.867988 (+ 92us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:38:14.870511 (+ 2523us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 15:38:14.870517 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 15:38:14.870523 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 15:38:14.870555 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0809 15:38:14.872306 (+ 1751us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 15:38:14.872958 (+ 652us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 15:38:14.872961 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 15:38:14.872963 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 15:38:14.873047 (+ 84us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 15:38:14.873475 (+ 428us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 15:38:14.873479 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 15:38:14.873480 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 15:38:14.873722 (+ 242us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 15:38:14.873826 (+ 104us) server_negotiation.cc:1035] Waiting for connection context
0809 15:38:14.874390 (+ 564us) server_negotiation.cc:299] Negotiation successful
0809 15:38:14.874534 (+ 144us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":52}
I20260809 15:38:14.875502 17967 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:38:14.862644 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.89.254:39401 (local address 127.0.0.1:51664)
0809 15:38:14.863095 (+ 451us) negotiation.cc:107] Waiting for socket to connect
0809 15:38:14.863140 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0809 15:38:14.863837 (+ 697us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 15:38:14.864901 (+ 1064us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 15:38:14.864905 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 15:38:14.865179 (+ 274us) client_negotiation.cc:189] Negotiated authn=SASL
0809 15:38:14.865321 (+ 142us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:38:14.865330 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:38:14.866343 (+ 1013us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:38:14.866346 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:38:14.867457 (+ 1111us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:38:14.867466 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:38:14.868056 (+ 590us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:38:14.868060 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:38:14.868190 (+ 130us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:38:14.868673 (+ 483us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 15:38:14.868696 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0809 15:38:14.870352 (+ 1656us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 15:38:14.872424 (+ 2072us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 15:38:14.872434 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 15:38:14.872442 (+ 8us) client_negotiation.cc:760] calling sasl_client_step()
0809 15:38:14.872809 (+ 367us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 15:38:14.873156 (+ 347us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 15:38:14.873160 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 15:38:14.873162 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 15:38:14.873352 (+ 190us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 15:38:14.873839 (+ 487us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 15:38:14.873844 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 15:38:14.874067 (+ 223us) client_negotiation.cc:769] Sending connection context
0809 15:38:14.874310 (+ 243us) client_negotiation.cc:240] Negotiation successful
0809 15:38:14.874625 (+ 315us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":251,"spinlock_wait_cycles":1408,"thread_start_us":129,"threads_started":1}
I20260809 15:38:14.875505 17968 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:38:14.862645 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53434 (local address 127.3.89.252:43425)
0809 15:38:14.863082 (+ 437us) server_negotiation.cc:206] Beginning negotiation
0809 15:38:14.863101 (+ 19us) server_negotiation.cc:399] Waiting for connection header
0809 15:38:14.863132 (+ 31us) server_negotiation.cc:407] Connection header received
0809 15:38:14.863819 (+ 687us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 15:38:14.863835 (+ 16us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 15:38:14.864049 (+ 214us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 15:38:14.864211 (+ 162us) server_negotiation.cc:226] Negotiated authn=SASL
0809 15:38:14.864887 (+ 676us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:38:14.866041 (+ 1154us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:38:14.867330 (+ 1289us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:38:14.867791 (+ 461us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:38:14.867960 (+ 169us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:38:14.871047 (+ 3087us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 15:38:14.871052 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 15:38:14.871060 (+ 8us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 15:38:14.871080 (+ 20us) server_negotiation.cc:922] calling sasl_server_start()
0809 15:38:14.872776 (+ 1696us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 15:38:14.873279 (+ 503us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 15:38:14.873284 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 15:38:14.873286 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 15:38:14.873355 (+ 69us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 15:38:14.873670 (+ 315us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 15:38:14.873673 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 15:38:14.873675 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 15:38:14.874048 (+ 373us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 15:38:14.874219 (+ 171us) server_negotiation.cc:1035] Waiting for connection context
0809 15:38:14.874522 (+ 303us) server_negotiation.cc:299] Negotiation successful
0809 15:38:14.874682 (+ 160us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":32,"server-negotiator.queue_time_us":324,"spinlock_wait_cycles":896,"thread_start_us":178,"threads_started":1}
I20260809 15:38:14.876766 17964 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.3.89.253" port: 33211 } has no permanent_uuid. Determining permanent_uuid...
Aug 09 15:38:14 dist-test-slave-2w3w krb5kdc[17720](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786289894, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.89.252@KRBTEST.COM for kudu/127.3.89.253@KRBTEST.COM
I20260809 15:38:14.881745 17807 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.3.89.252" port: 43425 } attempt: 3
Aug 09 15:38:14 dist-test-slave-2w3w krb5kdc[17720](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786289894, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.3.89.254@KRBTEST.COM
I20260809 15:38:14.885222 17884 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.3.89.252" port: 43425 } attempt: 3
I20260809 15:38:14.886034 17889 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:38:14.877389 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39856 (local address 127.3.89.253:33211)
0809 15:38:14.877535 (+ 146us) server_negotiation.cc:206] Beginning negotiation
0809 15:38:14.877540 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 15:38:14.877557 (+ 17us) server_negotiation.cc:407] Connection header received
0809 15:38:14.877604 (+ 47us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 15:38:14.877609 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 15:38:14.877682 (+ 73us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 15:38:14.877765 (+ 83us) server_negotiation.cc:226] Negotiated authn=SASL
0809 15:38:14.878176 (+ 411us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:38:14.878687 (+ 511us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:38:14.879552 (+ 865us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:38:14.879797 (+ 245us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:38:14.879910 (+ 113us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:38:14.882009 (+ 2099us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 15:38:14.882015 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 15:38:14.882021 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 15:38:14.882053 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0809 15:38:14.883834 (+ 1781us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 15:38:14.884566 (+ 732us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 15:38:14.884570 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 15:38:14.884572 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 15:38:14.884663 (+ 91us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 15:38:14.884979 (+ 316us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 15:38:14.884982 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 15:38:14.884984 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 15:38:14.885266 (+ 282us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 15:38:14.885340 (+ 74us) server_negotiation.cc:1035] Waiting for connection context
0809 15:38:14.885735 (+ 395us) server_negotiation.cc:299] Negotiation successful
0809 15:38:14.885859 (+ 124us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":44}
I20260809 15:38:14.886499 17967 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:38:14.877241 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.89.253:33211 (local address 127.0.0.1:39856)
0809 15:38:14.877338 (+ 97us) negotiation.cc:107] Waiting for socket to connect
0809 15:38:14.877352 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0809 15:38:14.877423 (+ 71us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 15:38:14.877779 (+ 356us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 15:38:14.877782 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 15:38:14.877948 (+ 166us) client_negotiation.cc:189] Negotiated authn=SASL
0809 15:38:14.878047 (+ 99us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:38:14.878053 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:38:14.878802 (+ 749us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:38:14.878807 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:38:14.879411 (+ 604us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:38:14.879420 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:38:14.879922 (+ 502us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:38:14.879925 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:38:14.880018 (+ 93us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:38:14.880475 (+ 457us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 15:38:14.880490 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0809 15:38:14.881854 (+ 1364us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 15:38:14.883952 (+ 2098us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 15:38:14.884156 (+ 204us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 15:38:14.884159 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0809 15:38:14.884438 (+ 279us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 15:38:14.884771 (+ 333us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 15:38:14.884774 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 15:38:14.884776 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 15:38:14.884865 (+ 89us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 15:38:14.885420 (+ 555us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 15:38:14.885423 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 15:38:14.885534 (+ 111us) client_negotiation.cc:769] Sending connection context
0809 15:38:14.886167 (+ 633us) client_negotiation.cc:240] Negotiation successful
0809 15:38:14.886303 (+ 136us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":30}
I20260809 15:38:14.887189 17964 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.3.89.252" port: 43425 } has no permanent_uuid. Determining permanent_uuid...
Aug 09 15:38:14 dist-test-slave-2w3w krb5kdc[17720](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786289894, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.89.254@KRBTEST.COM for kudu/127.3.89.252@KRBTEST.COM
I20260809 15:38:14.888041 17810 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:38:14.877645 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51680 (local address 127.3.89.254:39401)
0809 15:38:14.877752 (+ 107us) server_negotiation.cc:206] Beginning negotiation
0809 15:38:14.877758 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 15:38:14.877771 (+ 13us) server_negotiation.cc:407] Connection header received
0809 15:38:14.877887 (+ 116us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 15:38:14.877890 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 15:38:14.877924 (+ 34us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 15:38:14.878018 (+ 94us) server_negotiation.cc:226] Negotiated authn=SASL
0809 15:38:14.878608 (+ 590us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:38:14.879154 (+ 546us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:38:14.879922 (+ 768us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:38:14.880137 (+ 215us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:38:14.880296 (+ 159us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:38:14.882887 (+ 2591us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 15:38:14.882893 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 15:38:14.883158 (+ 265us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 15:38:14.883188 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0809 15:38:14.884799 (+ 1611us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 15:38:14.886224 (+ 1425us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 15:38:14.886228 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 15:38:14.886229 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 15:38:14.886313 (+ 84us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 15:38:14.886643 (+ 330us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 15:38:14.886649 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 15:38:14.886654 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0809 15:38:14.886864 (+ 210us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 15:38:14.887094 (+ 230us) server_negotiation.cc:1035] Waiting for connection context
0809 15:38:14.887755 (+ 661us) server_negotiation.cc:299] Negotiation successful
0809 15:38:14.887870 (+ 115us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":42}
I20260809 15:38:14.891789 17809 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:38:14.882139 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.89.252:43425 (local address 127.0.0.1:53446)
0809 15:38:14.882366 (+ 227us) negotiation.cc:107] Waiting for socket to connect
0809 15:38:14.882380 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0809 15:38:14.882447 (+ 67us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 15:38:14.882906 (+ 459us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 15:38:14.882909 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 15:38:14.883119 (+ 210us) client_negotiation.cc:189] Negotiated authn=SASL
0809 15:38:14.883215 (+ 96us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:38:14.883222 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:38:14.884241 (+ 1019us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:38:14.884246 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:38:14.885005 (+ 759us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:38:14.885017 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:38:14.885611 (+ 594us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:38:14.885614 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:38:14.885715 (+ 101us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:38:14.886340 (+ 625us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 15:38:14.886354 (+ 14us) client_negotiation.cc:656] calling sasl_client_start()
0809 15:38:14.888198 (+ 1844us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 15:38:14.890217 (+ 2019us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 15:38:14.890222 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 15:38:14.890225 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0809 15:38:14.890498 (+ 273us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 15:38:14.890806 (+ 308us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 15:38:14.890809 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 15:38:14.890810 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0809 15:38:14.890896 (+ 86us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 15:38:14.891328 (+ 432us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 15:38:14.891331 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 15:38:14.891407 (+ 76us) client_negotiation.cc:769] Sending connection context
0809 15:38:14.891513 (+ 106us) client_negotiation.cc:240] Negotiation successful
0809 15:38:14.891643 (+ 130us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":157}
I20260809 15:38:14.892307 17968 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:38:14.882610 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53446 (local address 127.3.89.252:43425)
0809 15:38:14.882716 (+ 106us) server_negotiation.cc:206] Beginning negotiation
0809 15:38:14.882719 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0809 15:38:14.882731 (+ 12us) server_negotiation.cc:407] Connection header received
0809 15:38:14.882762 (+ 31us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 15:38:14.882766 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 15:38:14.882813 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 15:38:14.882884 (+ 71us) server_negotiation.cc:226] Negotiated authn=SASL
0809 15:38:14.883356 (+ 472us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:38:14.884108 (+ 752us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:38:14.885127 (+ 1019us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:38:14.885490 (+ 363us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:38:14.885599 (+ 109us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:38:14.888339 (+ 2740us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 15:38:14.888342 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 15:38:14.888344 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 15:38:14.888381 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0809 15:38:14.890094 (+ 1713us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 15:38:14.890620 (+ 526us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 15:38:14.890622 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 15:38:14.890624 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 15:38:14.890705 (+ 81us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 15:38:14.891028 (+ 323us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 15:38:14.891031 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 15:38:14.891032 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 15:38:14.891207 (+ 175us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 15:38:14.891288 (+ 81us) server_negotiation.cc:1035] Waiting for connection context
0809 15:38:14.892067 (+ 779us) server_negotiation.cc:299] Negotiation successful
0809 15:38:14.892167 (+ 100us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":44}
Aug 09 15:38:14 dist-test-slave-2w3w krb5kdc[17720](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786289894, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.89.253@KRBTEST.COM for kudu/127.3.89.252@KRBTEST.COM
Aug 09 15:38:14 dist-test-slave-2w3w krb5kdc[17720](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786289894, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.89.252@KRBTEST.COM for kudu/127.3.89.252@KRBTEST.COM
Aug 09 15:38:14 dist-test-slave-2w3w krb5kdc[17720](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786289894, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.3.89.253@KRBTEST.COM
I20260809 15:38:14.896669 17807 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 16412a81332747479bfb62d9280280c3: Bootstrap starting.
I20260809 15:38:14.898119 17967 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:38:14.887600 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.89.252:43425 (local address 127.0.0.1:53472)
0809 15:38:14.888281 (+ 681us) negotiation.cc:107] Waiting for socket to connect
0809 15:38:14.888295 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0809 15:38:14.888360 (+ 65us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 15:38:14.888993 (+ 633us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 15:38:14.888995 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 15:38:14.889165 (+ 170us) client_negotiation.cc:189] Negotiated authn=SASL
0809 15:38:14.889262 (+ 97us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:38:14.889270 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:38:14.890328 (+ 1058us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:38:14.890333 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:38:14.891099 (+ 766us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:38:14.891111 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:38:14.891764 (+ 653us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:38:14.891767 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:38:14.892104 (+ 337us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:38:14.892621 (+ 517us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 15:38:14.892636 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0809 15:38:14.894047 (+ 1411us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 15:38:14.896342 (+ 2295us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 15:38:14.896347 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 15:38:14.896351 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0809 15:38:14.896903 (+ 552us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 15:38:14.897181 (+ 278us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 15:38:14.897183 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 15:38:14.897185 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 15:38:14.897259 (+ 74us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 15:38:14.897624 (+ 365us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 15:38:14.897627 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 15:38:14.897710 (+ 83us) client_negotiation.cc:769] Sending connection context
0809 15:38:14.897817 (+ 107us) client_negotiation.cc:240] Negotiation successful
0809 15:38:14.897938 (+ 121us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":597,"thread_start_us":84,"threads_started":1}
I20260809 15:38:14.898700 17971 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:38:14.888019 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53472 (local address 127.3.89.252:43425)
0809 15:38:14.888659 (+ 640us) server_negotiation.cc:206] Beginning negotiation
0809 15:38:14.888668 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0809 15:38:14.888681 (+ 13us) server_negotiation.cc:407] Connection header received
0809 15:38:14.888714 (+ 33us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 15:38:14.888719 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 15:38:14.888792 (+ 73us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 15:38:14.888876 (+ 84us) server_negotiation.cc:226] Negotiated authn=SASL
0809 15:38:14.889385 (+ 509us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:38:14.890211 (+ 826us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:38:14.891217 (+ 1006us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:38:14.891582 (+ 365us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:38:14.891675 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:38:14.894176 (+ 2501us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 15:38:14.894182 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 15:38:14.894188 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 15:38:14.894216 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0809 15:38:14.896226 (+ 2010us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 15:38:14.897000 (+ 774us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 15:38:14.897002 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 15:38:14.897004 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 15:38:14.897084 (+ 80us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 15:38:14.897354 (+ 270us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 15:38:14.897359 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 15:38:14.897361 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 15:38:14.897518 (+ 157us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 15:38:14.897624 (+ 106us) server_negotiation.cc:1035] Waiting for connection context
0809 15:38:14.898424 (+ 800us) server_negotiation.cc:299] Negotiation successful
0809 15:38:14.898528 (+ 104us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":573,"thread_start_us":110,"threads_started":1}
I20260809 15:38:14.898892 17886 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:38:14.885572 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.89.252:43425 (local address 127.0.0.1:53458)
0809 15:38:14.885752 (+ 180us) negotiation.cc:107] Waiting for socket to connect
0809 15:38:14.885765 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0809 15:38:14.885831 (+ 66us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 15:38:14.886625 (+ 794us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 15:38:14.886628 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 15:38:14.886795 (+ 167us) client_negotiation.cc:189] Negotiated authn=SASL
0809 15:38:14.886918 (+ 123us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:38:14.886928 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:38:14.889426 (+ 2498us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:38:14.889429 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:38:14.890241 (+ 812us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:38:14.890254 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:38:14.890841 (+ 587us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:38:14.890845 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:38:14.890926 (+ 81us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:38:14.891389 (+ 463us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 15:38:14.891403 (+ 14us) client_negotiation.cc:656] calling sasl_client_start()
0809 15:38:14.893724 (+ 2321us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 15:38:14.897003 (+ 3279us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 15:38:14.897006 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 15:38:14.897008 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 15:38:14.897534 (+ 526us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 15:38:14.897851 (+ 317us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 15:38:14.897854 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 15:38:14.897856 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 15:38:14.897946 (+ 90us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 15:38:14.898391 (+ 445us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 15:38:14.898394 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 15:38:14.898495 (+ 101us) client_negotiation.cc:769] Sending connection context
0809 15:38:14.898586 (+ 91us) client_negotiation.cc:240] Negotiation successful
0809 15:38:14.898703 (+ 117us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":119,"spinlock_wait_cycles":6272}
I20260809 15:38:14.899484 17889 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:38:14.889321 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39860 (local address 127.3.89.253:33211)
0809 15:38:14.889420 (+ 99us) server_negotiation.cc:206] Beginning negotiation
0809 15:38:14.889424 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 15:38:14.889609 (+ 185us) server_negotiation.cc:407] Connection header received
0809 15:38:14.889784 (+ 175us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 15:38:14.889788 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 15:38:14.889833 (+ 45us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 15:38:14.889887 (+ 54us) server_negotiation.cc:226] Negotiated authn=SASL
0809 15:38:14.890743 (+ 856us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:38:14.891462 (+ 719us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:38:14.892467 (+ 1005us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:38:14.892755 (+ 288us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:38:14.892837 (+ 82us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:38:14.895715 (+ 2878us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 15:38:14.895719 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 15:38:14.895721 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 15:38:14.895745 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0809 15:38:14.897303 (+ 1558us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 15:38:14.898013 (+ 710us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 15:38:14.898018 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 15:38:14.898021 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 15:38:14.898109 (+ 88us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 15:38:14.898429 (+ 320us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 15:38:14.898432 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 15:38:14.898434 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 15:38:14.898622 (+ 188us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 15:38:14.898806 (+ 184us) server_negotiation.cc:1035] Waiting for connection context
0809 15:38:14.899233 (+ 427us) server_negotiation.cc:299] Negotiation successful
0809 15:38:14.899337 (+ 104us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":37}
I20260809 15:38:14.899691 17969 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:38:14.886013 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53458 (local address 127.3.89.252:43425)
0809 15:38:14.886368 (+ 355us) server_negotiation.cc:206] Beginning negotiation
0809 15:38:14.886378 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0809 15:38:14.886393 (+ 15us) server_negotiation.cc:407] Connection header received
0809 15:38:14.886435 (+ 42us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 15:38:14.886438 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 15:38:14.886479 (+ 41us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 15:38:14.887072 (+ 593us) server_negotiation.cc:226] Negotiated authn=SASL
0809 15:38:14.887171 (+ 99us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:38:14.889080 (+ 1909us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:38:14.890383 (+ 1303us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:38:14.890721 (+ 338us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:38:14.893854 (+ 3133us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:38:14.893950 (+ 96us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 15:38:14.893952 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 15:38:14.893956 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 15:38:14.893979 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0809 15:38:14.896879 (+ 2900us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 15:38:14.897657 (+ 778us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 15:38:14.897660 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 15:38:14.897662 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 15:38:14.897734 (+ 72us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 15:38:14.898060 (+ 326us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 15:38:14.898063 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 15:38:14.898065 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 15:38:14.898266 (+ 201us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 15:38:14.899173 (+ 907us) server_negotiation.cc:1035] Waiting for connection context
0809 15:38:14.899245 (+ 72us) server_negotiation.cc:299] Negotiation successful
0809 15:38:14.899371 (+ 126us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":289,"thread_start_us":118,"threads_started":1}
I20260809 15:38:14.899811 17807 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 16412a81332747479bfb62d9280280c3: Neither blocks nor log segments found. Creating new log.
I20260809 15:38:14.900867 17807 log.cc:824] T 00000000000000000000000000000000 P 16412a81332747479bfb62d9280280c3: Log is configured to *not* fsync() on all Append() calls
I20260809 15:38:14.902482 17884 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 78e16fb3cd5f4817a798aaf26d28231f: Bootstrap starting.
I20260809 15:38:14.902738 17807 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 16412a81332747479bfb62d9280280c3: No bootstrap required, opened a new log
I20260809 15:38:14.903378 17964 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c835f885d12d4b1fbcf32bf63fa99c30: Bootstrap starting.
I20260809 15:38:14.904969 17884 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 78e16fb3cd5f4817a798aaf26d28231f: Neither blocks nor log segments found. Creating new log.
I20260809 15:38:14.905651 17964 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P c835f885d12d4b1fbcf32bf63fa99c30: Neither blocks nor log segments found. Creating new log.
I20260809 15:38:14.905773 17884 log.cc:824] T 00000000000000000000000000000000 P 78e16fb3cd5f4817a798aaf26d28231f: Log is configured to *not* fsync() on all Append() calls
I20260809 15:38:14.906141 17807 raft_consensus.cc:348] T 00000000000000000000000000000000 P 16412a81332747479bfb62d9280280c3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "16412a81332747479bfb62d9280280c3" member_type: VOTER last_known_addr { host: "127.3.89.254" port: 39401 } } peers { permanent_uuid: "78e16fb3cd5f4817a798aaf26d28231f" member_type: VOTER last_known_addr { host: "127.3.89.253" port: 33211 } } peers { permanent_uuid: "c835f885d12d4b1fbcf32bf63fa99c30" member_type: VOTER last_known_addr { host: "127.3.89.252" port: 43425 } }
I20260809 15:38:14.906394 17807 raft_consensus.cc:374] T 00000000000000000000000000000000 P 16412a81332747479bfb62d9280280c3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 15:38:14.906450 17807 raft_consensus.cc:729] T 00000000000000000000000000000000 P 16412a81332747479bfb62d9280280c3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 16412a81332747479bfb62d9280280c3, State: Initialized, Role: FOLLOWER
I20260809 15:38:14.906452 17964 log.cc:824] T 00000000000000000000000000000000 P c835f885d12d4b1fbcf32bf63fa99c30: Log is configured to *not* fsync() on all Append() calls
I20260809 15:38:14.906934 17807 consensus_queue.cc:260] T 00000000000000000000000000000000 P 16412a81332747479bfb62d9280280c3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "16412a81332747479bfb62d9280280c3" member_type: VOTER last_known_addr { host: "127.3.89.254" port: 39401 } } peers { permanent_uuid: "78e16fb3cd5f4817a798aaf26d28231f" member_type: VOTER last_known_addr { host: "127.3.89.253" port: 33211 } } peers { permanent_uuid: "c835f885d12d4b1fbcf32bf63fa99c30" member_type: VOTER last_known_addr { host: "127.3.89.252" port: 43425 } }
I20260809 15:38:14.908161 17884 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 78e16fb3cd5f4817a798aaf26d28231f: No bootstrap required, opened a new log
I20260809 15:38:14.908270 17807 sys_catalog.cc:564] T 00000000000000000000000000000000 P 16412a81332747479bfb62d9280280c3 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 15:38:14.908447 17964 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c835f885d12d4b1fbcf32bf63fa99c30: No bootstrap required, opened a new log
I20260809 15:38:14.908874 3431 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
W20260809 15:38:14.911168 17755 tablet.cc:2369] T 00000000000000000000000000000000 P 16412a81332747479bfb62d9280280c3: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260809 15:38:14.912048 17975 sys_catalog.cc:455] T 00000000000000000000000000000000 P 16412a81332747479bfb62d9280280c3 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "16412a81332747479bfb62d9280280c3" member_type: VOTER last_known_addr { host: "127.3.89.254" port: 39401 } } peers { permanent_uuid: "78e16fb3cd5f4817a798aaf26d28231f" member_type: VOTER last_known_addr { host: "127.3.89.253" port: 33211 } } peers { permanent_uuid: "c835f885d12d4b1fbcf32bf63fa99c30" member_type: VOTER last_known_addr { host: "127.3.89.252" port: 43425 } } }
I20260809 15:38:14.912168 17975 sys_catalog.cc:458] T 00000000000000000000000000000000 P 16412a81332747479bfb62d9280280c3 [sys.catalog]: This master's current role is: FOLLOWER
I20260809 15:38:14.913164 17884 raft_consensus.cc:348] T 00000000000000000000000000000000 P 78e16fb3cd5f4817a798aaf26d28231f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "16412a81332747479bfb62d9280280c3" member_type: VOTER last_known_addr { host: "127.3.89.254" port: 39401 } } peers { permanent_uuid: "78e16fb3cd5f4817a798aaf26d28231f" member_type: VOTER last_known_addr { host: "127.3.89.253" port: 33211 } } peers { permanent_uuid: "c835f885d12d4b1fbcf32bf63fa99c30" member_type: VOTER last_known_addr { host: "127.3.89.252" port: 43425 } }
I20260809 15:38:14.913385 17964 raft_consensus.cc:348] T 00000000000000000000000000000000 P c835f885d12d4b1fbcf32bf63fa99c30 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "16412a81332747479bfb62d9280280c3" member_type: VOTER last_known_addr { host: "127.3.89.254" port: 39401 } } peers { permanent_uuid: "78e16fb3cd5f4817a798aaf26d28231f" member_type: VOTER last_known_addr { host: "127.3.89.253" port: 33211 } } peers { permanent_uuid: "c835f885d12d4b1fbcf32bf63fa99c30" member_type: VOTER last_known_addr { host: "127.3.89.252" port: 43425 } }
I20260809 15:38:14.913625 17964 raft_consensus.cc:374] T 00000000000000000000000000000000 P c835f885d12d4b1fbcf32bf63fa99c30 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 15:38:14.913683 17964 raft_consensus.cc:729] T 00000000000000000000000000000000 P c835f885d12d4b1fbcf32bf63fa99c30 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c835f885d12d4b1fbcf32bf63fa99c30, State: Initialized, Role: FOLLOWER
I20260809 15:38:14.914135 17964 consensus_queue.cc:260] T 00000000000000000000000000000000 P c835f885d12d4b1fbcf32bf63fa99c30 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "16412a81332747479bfb62d9280280c3" member_type: VOTER last_known_addr { host: "127.3.89.254" port: 39401 } } peers { permanent_uuid: "78e16fb3cd5f4817a798aaf26d28231f" member_type: VOTER last_known_addr { host: "127.3.89.253" port: 33211 } } peers { permanent_uuid: "c835f885d12d4b1fbcf32bf63fa99c30" member_type: VOTER last_known_addr { host: "127.3.89.252" port: 43425 } }
I20260809 15:38:14.915547 17964 sys_catalog.cc:564] T 00000000000000000000000000000000 P c835f885d12d4b1fbcf32bf63fa99c30 [sys.catalog]: configured and running, proceeding with master startup.
W20260809 15:38:14.920732 18006 catalog_manager.cc:1558] T 00000000000000000000000000000000 P c835f885d12d4b1fbcf32bf63fa99c30: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260809 15:38:14.920804 18006 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260809 15:38:14.921057 17994 sys_catalog.cc:455] T 00000000000000000000000000000000 P c835f885d12d4b1fbcf32bf63fa99c30 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "16412a81332747479bfb62d9280280c3" member_type: VOTER last_known_addr { host: "127.3.89.254" port: 39401 } } peers { permanent_uuid: "78e16fb3cd5f4817a798aaf26d28231f" member_type: VOTER last_known_addr { host: "127.3.89.253" port: 33211 } } peers { permanent_uuid: "c835f885d12d4b1fbcf32bf63fa99c30" member_type: VOTER last_known_addr { host: "127.3.89.252" port: 43425 } } }
I20260809 15:38:14.921169 17994 sys_catalog.cc:458] T 00000000000000000000000000000000 P c835f885d12d4b1fbcf32bf63fa99c30 [sys.catalog]: This master's current role is: FOLLOWER
I20260809 15:38:14.914685 17734 master_runner.cc:186] Error getting master registration for 127.3.89.253:33211: OK, instance_id { permanent_uuid: "78e16fb3cd5f4817a798aaf26d28231f" instance_seqno: 1786289894631584 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260809 15:38:14.921963 17734 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260809 15:38:14.913435 17884 raft_consensus.cc:374] T 00000000000000000000000000000000 P 78e16fb3cd5f4817a798aaf26d28231f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 15:38:14.922097 17884 raft_consensus.cc:729] T 00000000000000000000000000000000 P 78e16fb3cd5f4817a798aaf26d28231f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 78e16fb3cd5f4817a798aaf26d28231f, State: Initialized, Role: FOLLOWER
I20260809 15:38:14.922806 17884 consensus_queue.cc:260] T 00000000000000000000000000000000 P 78e16fb3cd5f4817a798aaf26d28231f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "16412a81332747479bfb62d9280280c3" member_type: VOTER last_known_addr { host: "127.3.89.254" port: 39401 } } peers { permanent_uuid: "78e16fb3cd5f4817a798aaf26d28231f" member_type: VOTER last_known_addr { host: "127.3.89.253" port: 33211 } } peers { permanent_uuid: "c835f885d12d4b1fbcf32bf63fa99c30" member_type: VOTER last_known_addr { host: "127.3.89.252" port: 43425 } }
I20260809 15:38:14.923700 17889 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:38:14.911243 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39868 (local address 127.3.89.253:33211)
0809 15:38:14.911347 (+ 104us) server_negotiation.cc:206] Beginning negotiation
0809 15:38:14.911351 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 15:38:14.913847 (+ 2496us) server_negotiation.cc:407] Connection header received
0809 15:38:14.914031 (+ 184us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 15:38:14.914034 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 15:38:14.914078 (+ 44us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 15:38:14.914125 (+ 47us) server_negotiation.cc:226] Negotiated authn=SASL
0809 15:38:14.915168 (+ 1043us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:38:14.915918 (+ 750us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:38:14.917180 (+ 1262us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:38:14.917582 (+ 402us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:38:14.917662 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:38:14.919584 (+ 1922us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 15:38:14.919588 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 15:38:14.919590 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 15:38:14.919614 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0809 15:38:14.921369 (+ 1755us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 15:38:14.921883 (+ 514us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 15:38:14.921889 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 15:38:14.921891 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 15:38:14.921986 (+ 95us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 15:38:14.922442 (+ 456us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 15:38:14.922447 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 15:38:14.922452 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0809 15:38:14.922667 (+ 215us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 15:38:14.922764 (+ 97us) server_negotiation.cc:1035] Waiting for connection context
0809 15:38:14.923442 (+ 678us) server_negotiation.cc:299] Negotiation successful
0809 15:38:14.923557 (+ 115us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":42}
I20260809 15:38:14.924993 17884 sys_catalog.cc:564] T 00000000000000000000000000000000 P 78e16fb3cd5f4817a798aaf26d28231f [sys.catalog]: configured and running, proceeding with master startup.
I20260809 15:38:14.922747 17892 master_runner.cc:186] Error getting master registration for 127.3.89.253:33211: OK, instance_id { permanent_uuid: "78e16fb3cd5f4817a798aaf26d28231f" instance_seqno: 1786289894631584 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260809 15:38:14.926358 17892 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260809 15:38:14.927198 18008 sys_catalog.cc:455] T 00000000000000000000000000000000 P 78e16fb3cd5f4817a798aaf26d28231f [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "16412a81332747479bfb62d9280280c3" member_type: VOTER last_known_addr { host: "127.3.89.254" port: 39401 } } peers { permanent_uuid: "78e16fb3cd5f4817a798aaf26d28231f" member_type: VOTER last_known_addr { host: "127.3.89.253" port: 33211 } } peers { permanent_uuid: "c835f885d12d4b1fbcf32bf63fa99c30" member_type: VOTER last_known_addr { host: "127.3.89.252" port: 43425 } } }
I20260809 15:38:14.927395 18008 sys_catalog.cc:458] T 00000000000000000000000000000000 P 78e16fb3cd5f4817a798aaf26d28231f [sys.catalog]: This master's current role is: FOLLOWER
I20260809 15:38:14.929364 17969 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:38:14.911847 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53478 (local address 127.3.89.252:43425)
0809 15:38:14.912023 (+ 176us) server_negotiation.cc:206] Beginning negotiation
0809 15:38:14.912027 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 15:38:14.913902 (+ 1875us) server_negotiation.cc:407] Connection header received
0809 15:38:14.914088 (+ 186us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 15:38:14.914091 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 15:38:14.914129 (+ 38us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 15:38:14.914177 (+ 48us) server_negotiation.cc:226] Negotiated authn=SASL
0809 15:38:14.920355 (+ 6178us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:38:14.921087 (+ 732us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:38:14.922191 (+ 1104us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:38:14.922556 (+ 365us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:38:14.922631 (+ 75us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:38:14.925456 (+ 2825us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 15:38:14.925459 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 15:38:14.925462 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 15:38:14.925486 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0809 15:38:14.927313 (+ 1827us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 15:38:14.927877 (+ 564us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 15:38:14.927881 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 15:38:14.927882 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 15:38:14.927968 (+ 86us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 15:38:14.928303 (+ 335us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 15:38:14.928310 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 15:38:14.928314 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 15:38:14.928560 (+ 246us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 15:38:14.928681 (+ 121us) server_negotiation.cc:1035] Waiting for connection context
0809 15:38:14.929098 (+ 417us) server_negotiation.cc:299] Negotiation successful
0809 15:38:14.929214 (+ 116us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":45}
W20260809 15:38:14.932039 18019 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 78e16fb3cd5f4817a798aaf26d28231f: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260809 15:38:14.932173 18019 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260809 15:38:14.932900 17810 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:38:14.912467 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51684 (local address 127.3.89.254:39401)
0809 15:38:14.913346 (+ 879us) server_negotiation.cc:206] Beginning negotiation
0809 15:38:14.913350 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 15:38:14.915137 (+ 1787us) server_negotiation.cc:407] Connection header received
0809 15:38:14.915303 (+ 166us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 15:38:14.915306 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 15:38:14.915347 (+ 41us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 15:38:14.915395 (+ 48us) server_negotiation.cc:226] Negotiated authn=SASL
0809 15:38:14.916360 (+ 965us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:38:14.917141 (+ 781us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:38:14.918302 (+ 1161us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:38:14.918608 (+ 306us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:38:14.918683 (+ 75us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:38:14.923872 (+ 5189us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 15:38:14.923876 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 15:38:14.923879 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 15:38:14.923924 (+ 45us) server_negotiation.cc:922] calling sasl_server_start()
0809 15:38:14.925653 (+ 1729us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 15:38:14.931399 (+ 5746us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 15:38:14.931403 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 15:38:14.931405 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 15:38:14.931500 (+ 95us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 15:38:14.931847 (+ 347us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 15:38:14.931850 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 15:38:14.931852 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 15:38:14.932081 (+ 229us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 15:38:14.932232 (+ 151us) server_negotiation.cc:1035] Waiting for connection context
0809 15:38:14.932600 (+ 368us) server_negotiation.cc:299] Negotiation successful
0809 15:38:14.932721 (+ 121us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":812}
I20260809 15:38:14.935432 17814 master_runner.cc:263] No leader master found from master 78e16fb3cd5f4817a798aaf26d28231f
I20260809 15:38:14.935504 17814 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
W20260809 15:38:14.935820 17988 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 16412a81332747479bfb62d9280280c3: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260809 15:38:14.935899 17988 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
W20260809 15:38:15.097435 17913 tablet.cc:2369] T 00000000000000000000000000000000 P c835f885d12d4b1fbcf32bf63fa99c30: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260809 15:38:15.128175 17835 tablet.cc:2369] T 00000000000000000000000000000000 P 78e16fb3cd5f4817a798aaf26d28231f: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260809 15:38:15.279716 18008 raft_consensus.cc:482] T 00000000000000000000000000000000 P 78e16fb3cd5f4817a798aaf26d28231f [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 15:38:15.279922 18008 raft_consensus.cc:504] T 00000000000000000000000000000000 P 78e16fb3cd5f4817a798aaf26d28231f [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "16412a81332747479bfb62d9280280c3" member_type: VOTER last_known_addr { host: "127.3.89.254" port: 39401 } } peers { permanent_uuid: "78e16fb3cd5f4817a798aaf26d28231f" member_type: VOTER last_known_addr { host: "127.3.89.253" port: 33211 } } peers { permanent_uuid: "c835f885d12d4b1fbcf32bf63fa99c30" member_type: VOTER last_known_addr { host: "127.3.89.252" port: 43425 } }
I20260809 15:38:15.280856 18008 leader_election.cc:288] T 00000000000000000000000000000000 P 78e16fb3cd5f4817a798aaf26d28231f [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 16412a81332747479bfb62d9280280c3 (127.3.89.254:39401), c835f885d12d4b1fbcf32bf63fa99c30 (127.3.89.252:43425)
I20260809 15:38:15.281395 17776 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "78e16fb3cd5f4817a798aaf26d28231f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "16412a81332747479bfb62d9280280c3" is_pre_election: true
I20260809 15:38:15.281395 17936 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "78e16fb3cd5f4817a798aaf26d28231f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c835f885d12d4b1fbcf32bf63fa99c30" is_pre_election: true
I20260809 15:38:15.281710 17936 raft_consensus.cc:2454] T 00000000000000000000000000000000 P c835f885d12d4b1fbcf32bf63fa99c30 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 78e16fb3cd5f4817a798aaf26d28231f in term 0.
I20260809 15:38:15.281710 17776 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 16412a81332747479bfb62d9280280c3 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 78e16fb3cd5f4817a798aaf26d28231f in term 0.
I20260809 15:38:15.282162 17830 leader_election.cc:302] T 00000000000000000000000000000000 P 78e16fb3cd5f4817a798aaf26d28231f [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 78e16fb3cd5f4817a798aaf26d28231f, c835f885d12d4b1fbcf32bf63fa99c30; no voters:
I20260809 15:38:15.282482 18008 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 78e16fb3cd5f4817a798aaf26d28231f [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260809 15:38:15.282568 18008 raft_consensus.cc:482] T 00000000000000000000000000000000 P 78e16fb3cd5f4817a798aaf26d28231f [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 15:38:15.282600 18008 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 78e16fb3cd5f4817a798aaf26d28231f [term 0 FOLLOWER]: Advancing to term 1
I20260809 15:38:15.283753 18008 raft_consensus.cc:504] T 00000000000000000000000000000000 P 78e16fb3cd5f4817a798aaf26d28231f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "16412a81332747479bfb62d9280280c3" member_type: VOTER last_known_addr { host: "127.3.89.254" port: 39401 } } peers { permanent_uuid: "78e16fb3cd5f4817a798aaf26d28231f" member_type: VOTER last_known_addr { host: "127.3.89.253" port: 33211 } } peers { permanent_uuid: "c835f885d12d4b1fbcf32bf63fa99c30" member_type: VOTER last_known_addr { host: "127.3.89.252" port: 43425 } }
I20260809 15:38:15.284085 18008 leader_election.cc:288] T 00000000000000000000000000000000 P 78e16fb3cd5f4817a798aaf26d28231f [CANDIDATE]: Term 1 election: Requested vote from peers 16412a81332747479bfb62d9280280c3 (127.3.89.254:39401), c835f885d12d4b1fbcf32bf63fa99c30 (127.3.89.252:43425)
I20260809 15:38:15.284423 17776 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "78e16fb3cd5f4817a798aaf26d28231f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "16412a81332747479bfb62d9280280c3"
I20260809 15:38:15.284425 17936 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "78e16fb3cd5f4817a798aaf26d28231f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c835f885d12d4b1fbcf32bf63fa99c30"
I20260809 15:38:15.284538 17936 raft_consensus.cc:3037] T 00000000000000000000000000000000 P c835f885d12d4b1fbcf32bf63fa99c30 [term 0 FOLLOWER]: Advancing to term 1
I20260809 15:38:15.284566 17776 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 16412a81332747479bfb62d9280280c3 [term 0 FOLLOWER]: Advancing to term 1
I20260809 15:38:15.285595 17776 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 16412a81332747479bfb62d9280280c3 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 78e16fb3cd5f4817a798aaf26d28231f in term 1.
I20260809 15:38:15.285596 17936 raft_consensus.cc:2454] T 00000000000000000000000000000000 P c835f885d12d4b1fbcf32bf63fa99c30 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 78e16fb3cd5f4817a798aaf26d28231f in term 1.
I20260809 15:38:15.285892 17831 leader_election.cc:302] T 00000000000000000000000000000000 P 78e16fb3cd5f4817a798aaf26d28231f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 16412a81332747479bfb62d9280280c3, 78e16fb3cd5f4817a798aaf26d28231f; no voters:
I20260809 15:38:15.286104 18008 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 78e16fb3cd5f4817a798aaf26d28231f [term 1 FOLLOWER]: Leader election won for term 1
I20260809 15:38:15.286291 18008 raft_consensus.cc:686] T 00000000000000000000000000000000 P 78e16fb3cd5f4817a798aaf26d28231f [term 1 LEADER]: Becoming Leader. State: Replica: 78e16fb3cd5f4817a798aaf26d28231f, State: Running, Role: LEADER
I20260809 15:38:15.286609 18008 consensus_queue.cc:237] T 00000000000000000000000000000000 P 78e16fb3cd5f4817a798aaf26d28231f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "16412a81332747479bfb62d9280280c3" member_type: VOTER last_known_addr { host: "127.3.89.254" port: 39401 } } peers { permanent_uuid: "78e16fb3cd5f4817a798aaf26d28231f" member_type: VOTER last_known_addr { host: "127.3.89.253" port: 33211 } } peers { permanent_uuid: "c835f885d12d4b1fbcf32bf63fa99c30" member_type: VOTER last_known_addr { host: "127.3.89.252" port: 43425 } }
I20260809 15:38:15.288597 18022 sys_catalog.cc:455] T 00000000000000000000000000000000 P 78e16fb3cd5f4817a798aaf26d28231f [sys.catalog]: SysCatalogTable state changed. Reason: New leader 78e16fb3cd5f4817a798aaf26d28231f. Latest consensus state: current_term: 1 leader_uuid: "78e16fb3cd5f4817a798aaf26d28231f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "16412a81332747479bfb62d9280280c3" member_type: VOTER last_known_addr { host: "127.3.89.254" port: 39401 } } peers { permanent_uuid: "78e16fb3cd5f4817a798aaf26d28231f" member_type: VOTER last_known_addr { host: "127.3.89.253" port: 33211 } } peers { permanent_uuid: "c835f885d12d4b1fbcf32bf63fa99c30" member_type: VOTER last_known_addr { host: "127.3.89.252" port: 43425 } } }
I20260809 15:38:15.288728 18022 sys_catalog.cc:458] T 00000000000000000000000000000000 P 78e16fb3cd5f4817a798aaf26d28231f [sys.catalog]: This master's current role is: LEADER
I20260809 15:38:15.289047 18024 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 15:38:15.289865 18024 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 15:38:15.294155 17776 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 16412a81332747479bfb62d9280280c3 [term 1 FOLLOWER]: Refusing update from remote peer 78e16fb3cd5f4817a798aaf26d28231f: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 15:38:15.294155 17936 raft_consensus.cc:1261] T 00000000000000000000000000000000 P c835f885d12d4b1fbcf32bf63fa99c30 [term 1 FOLLOWER]: Refusing update from remote peer 78e16fb3cd5f4817a798aaf26d28231f: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 15:38:15.294785 18022 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 78e16fb3cd5f4817a798aaf26d28231f [LEADER]: Connected to new peer: Peer: permanent_uuid: "c835f885d12d4b1fbcf32bf63fa99c30" member_type: VOTER last_known_addr { host: "127.3.89.252" port: 43425 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 15:38:15.294942 18008 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 78e16fb3cd5f4817a798aaf26d28231f [LEADER]: Connected to new peer: Peer: permanent_uuid: "16412a81332747479bfb62d9280280c3" member_type: VOTER last_known_addr { host: "127.3.89.254" port: 39401 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 15:38:15.298316 17975 sys_catalog.cc:455] T 00000000000000000000000000000000 P 16412a81332747479bfb62d9280280c3 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 78e16fb3cd5f4817a798aaf26d28231f. Latest consensus state: current_term: 1 leader_uuid: "78e16fb3cd5f4817a798aaf26d28231f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "16412a81332747479bfb62d9280280c3" member_type: VOTER last_known_addr { host: "127.3.89.254" port: 39401 } } peers { permanent_uuid: "78e16fb3cd5f4817a798aaf26d28231f" member_type: VOTER last_known_addr { host: "127.3.89.253" port: 33211 } } peers { permanent_uuid: "c835f885d12d4b1fbcf32bf63fa99c30" member_type: VOTER last_known_addr { host: "127.3.89.252" port: 43425 } } }
I20260809 15:38:15.298427 17975 sys_catalog.cc:458] T 00000000000000000000000000000000 P 16412a81332747479bfb62d9280280c3 [sys.catalog]: This master's current role is: FOLLOWER
I20260809 15:38:15.299865 18008 sys_catalog.cc:455] T 00000000000000000000000000000000 P 78e16fb3cd5f4817a798aaf26d28231f [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "78e16fb3cd5f4817a798aaf26d28231f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "16412a81332747479bfb62d9280280c3" member_type: VOTER last_known_addr { host: "127.3.89.254" port: 39401 } } peers { permanent_uuid: "78e16fb3cd5f4817a798aaf26d28231f" member_type: VOTER last_known_addr { host: "127.3.89.253" port: 33211 } } peers { permanent_uuid: "c835f885d12d4b1fbcf32bf63fa99c30" member_type: VOTER last_known_addr { host: "127.3.89.252" port: 43425 } } }
I20260809 15:38:15.299979 18008 sys_catalog.cc:458] T 00000000000000000000000000000000 P 78e16fb3cd5f4817a798aaf26d28231f [sys.catalog]: This master's current role is: LEADER
I20260809 15:38:15.300710 17975 sys_catalog.cc:455] T 00000000000000000000000000000000 P 16412a81332747479bfb62d9280280c3 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "78e16fb3cd5f4817a798aaf26d28231f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "16412a81332747479bfb62d9280280c3" member_type: VOTER last_known_addr { host: "127.3.89.254" port: 39401 } } peers { permanent_uuid: "78e16fb3cd5f4817a798aaf26d28231f" member_type: VOTER last_known_addr { host: "127.3.89.253" port: 33211 } } peers { permanent_uuid: "c835f885d12d4b1fbcf32bf63fa99c30" member_type: VOTER last_known_addr { host: "127.3.89.252" port: 43425 } } }
I20260809 15:38:15.300819 17975 sys_catalog.cc:458] T 00000000000000000000000000000000 P 16412a81332747479bfb62d9280280c3 [sys.catalog]: This master's current role is: FOLLOWER
I20260809 15:38:15.300853 17994 sys_catalog.cc:455] T 00000000000000000000000000000000 P c835f885d12d4b1fbcf32bf63fa99c30 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 78e16fb3cd5f4817a798aaf26d28231f. Latest consensus state: current_term: 1 leader_uuid: "78e16fb3cd5f4817a798aaf26d28231f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "16412a81332747479bfb62d9280280c3" member_type: VOTER last_known_addr { host: "127.3.89.254" port: 39401 } } peers { permanent_uuid: "78e16fb3cd5f4817a798aaf26d28231f" member_type: VOTER last_known_addr { host: "127.3.89.253" port: 33211 } } peers { permanent_uuid: "c835f885d12d4b1fbcf32bf63fa99c30" member_type: VOTER last_known_addr { host: "127.3.89.252" port: 43425 } } }
I20260809 15:38:15.300987 17994 sys_catalog.cc:458] T 00000000000000000000000000000000 P c835f885d12d4b1fbcf32bf63fa99c30 [sys.catalog]: This master's current role is: FOLLOWER
I20260809 15:38:15.301086 18026 mvcc.cc:205] Tried to move back new op lower bound from 7316643411119149056 to 7316643411096940544. Current Snapshot: MvccSnapshot[applied={T|T < 7316643411119149056}]
I20260809 15:38:15.301194 18025 mvcc.cc:205] Tried to move back new op lower bound from 7316643411119149056 to 7316643411096940544. Current Snapshot: MvccSnapshot[applied={T|T < 7316643411119149056}]
I20260809 15:38:15.301702 18024 catalog_manager.cc:1347] Generated new cluster ID: 10fb761a307f47c18a512858c5240e81
I20260809 15:38:15.301766 18024 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 15:38:15.304219 18022 sys_catalog.cc:455] T 00000000000000000000000000000000 P 78e16fb3cd5f4817a798aaf26d28231f [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "78e16fb3cd5f4817a798aaf26d28231f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "16412a81332747479bfb62d9280280c3" member_type: VOTER last_known_addr { host: "127.3.89.254" port: 39401 } } peers { permanent_uuid: "78e16fb3cd5f4817a798aaf26d28231f" member_type: VOTER last_known_addr { host: "127.3.89.253" port: 33211 } } peers { permanent_uuid: "c835f885d12d4b1fbcf32bf63fa99c30" member_type: VOTER last_known_addr { host: "127.3.89.252" port: 43425 } } }
I20260809 15:38:15.304333 18022 sys_catalog.cc:458] T 00000000000000000000000000000000 P 78e16fb3cd5f4817a798aaf26d28231f [sys.catalog]: This master's current role is: LEADER
I20260809 15:38:15.305038 17994 sys_catalog.cc:455] T 00000000000000000000000000000000 P c835f885d12d4b1fbcf32bf63fa99c30 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "78e16fb3cd5f4817a798aaf26d28231f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "16412a81332747479bfb62d9280280c3" member_type: VOTER last_known_addr { host: "127.3.89.254" port: 39401 } } peers { permanent_uuid: "78e16fb3cd5f4817a798aaf26d28231f" member_type: VOTER last_known_addr { host: "127.3.89.253" port: 33211 } } peers { permanent_uuid: "c835f885d12d4b1fbcf32bf63fa99c30" member_type: VOTER last_known_addr { host: "127.3.89.252" port: 43425 } } }
I20260809 15:38:15.305137 17994 sys_catalog.cc:458] T 00000000000000000000000000000000 P c835f885d12d4b1fbcf32bf63fa99c30 [sys.catalog]: This master's current role is: FOLLOWER
I20260809 15:38:15.311419 18024 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 15:38:15.312561 18024 catalog_manager.cc:1504] Loading token signing keys...
I20260809 15:38:15.321184 18024 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 78e16fb3cd5f4817a798aaf26d28231f: Generated new TSK 0
I20260809 15:38:15.321993 18024 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260809 15:38:15.922048 18006 catalog_manager.cc:1259] Loaded cluster ID: 10fb761a307f47c18a512858c5240e81
I20260809 15:38:15.922135 18006 catalog_manager.cc:1552] T 00000000000000000000000000000000 P c835f885d12d4b1fbcf32bf63fa99c30: loading cluster ID for follower catalog manager: success
I20260809 15:38:15.924010 18006 catalog_manager.cc:1574] T 00000000000000000000000000000000 P c835f885d12d4b1fbcf32bf63fa99c30: acquiring CA information for follower catalog manager: success
I20260809 15:38:15.925287 18006 catalog_manager.cc:1602] T 00000000000000000000000000000000 P c835f885d12d4b1fbcf32bf63fa99c30: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260809 15:38:15.936954 17988 catalog_manager.cc:1259] Loaded cluster ID: 10fb761a307f47c18a512858c5240e81
I20260809 15:38:15.937016 17988 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 16412a81332747479bfb62d9280280c3: loading cluster ID for follower catalog manager: success
I20260809 15:38:15.938561 17988 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 16412a81332747479bfb62d9280280c3: acquiring CA information for follower catalog manager: success
I20260809 15:38:15.939692 17988 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 16412a81332747479bfb62d9280280c3: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260809 15:38:16.393512 3431 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskm314YR/build/debug/bin/kudu with pid 17734
I20260809 15:38:16.401742 3431 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskm314YR/build/debug/bin/kudu with pid 17814
I20260809 15:38:16.408916 3431 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskm314YR/build/debug/bin/kudu with pid 17892
2026-08-09T15:38:16Z chronyd exiting
[ OK ] SecurityITest.IPKICACert (3406 ms)
[ RUN ] SecurityITest.IPKICACertWebServerEndPoints
Loading random data
Initializing database '/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786289732823074-3431-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 15:38:16 dist-test-slave-2w3w krb5kdc[18041](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 15:38:16 dist-test-slave-2w3w krb5kdc[18041](info): set up 2 sockets
Aug 09 15:38:16 dist-test-slave-2w3w krb5kdc[18041](info): commencing operation
krb5kdc: starting...
W20260809 15:38:17.498070 3431 mini_kdc.cc:121] Time spent starting KDC: real 1.067s user 0.000s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786289732823074-3431-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786289732823074-3431-0/krb5kdc/test-user.keytab.
Aug 09 15:38:17 dist-test-slave-2w3w krb5kdc[18041](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786289897, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T15:38:17Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T15:38:17Z Disabled control of system clock
WARNING: no policy specified for kudu/127.3.89.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.3.89.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.3.89.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786289732823074-3431-0/krb5kdc/kudu_127.3.89.254.keytab.
Entry for principal kudu/127.3.89.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786289732823074-3431-0/krb5kdc/kudu_127.3.89.254.keytab.
I20260809 15:38:17.626168 3431 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskm314YR/build/debug/bin/kudu
/tmp/dist-test-taskm314YR/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786289732823074-3431-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786289732823074-3431-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786289732823074-3431-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786289732823074-3431-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.3.89.254:38743
--webserver_interface=127.3.89.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786289732823074-3431-0/krb5kdc/kudu_127.3.89.254.keytab
--principal=kudu/127.3.89.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.3.89.212:33119
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.3.89.254:38743
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786289732823074-3431-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786289732823074-3431-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786289732823074-3431-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 15:38:17.733628 18054 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 15:38:17.733925 18054 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 15:38:17.734005 18054 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 15:38:17.737965 18054 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 15:38:17.738056 18054 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 15:38:17.738096 18054 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 15:38:17.738132 18054 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 15:38:17.738161 18054 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 15:38:17.743250 18054 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.3.89.212:33119
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786289732823074-3431-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786289732823074-3431-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.3.89.254:38743
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786289732823074-3431-0/krb5kdc/kudu_127.3.89.254.keytab
--principal=kudu/127.3.89.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.3.89.254:38743
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786289732823074-3431-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.3.89.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.18054
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786289732823074-3431-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 09 Aug 2026 14:46:55 UTC on ccycloud.quasar-yswkmf.root.comops.site
build id 4576540
I20260809 15:38:17.744257 18054 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 15:38:17.745210 18054 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 15:38:17.753407 18060 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 15:38:17.753507 18061 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 15:38:17.753441 18063 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 15:38:17.753507 18054 server_base.cc:1034] running on GCE node
I20260809 15:38:17.754315 18054 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 15:38:17.755440 18054 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 15:38:17.756635 18054 hybrid_clock.cc:648] HybridClock initialized: now 1786289897756620 us; error 35 us; skew 500 ppm
Aug 09 15:38:17 dist-test-slave-2w3w krb5kdc[18041](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786289897, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.89.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 15:38:17.759362 18054 init.cc:377] Logged in from keytab as kudu/127.3.89.254@KRBTEST.COM (short username kudu)
I20260809 15:38:17.760221 18054 webserver.cc:460] Webserver started at http://127.3.89.254:34947/ using document root <none> and password file <none>
I20260809 15:38:17.760722 18054 fs_manager.cc:359] Metadata directory not provided
I20260809 15:38:17.760792 18054 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 15:38:17.760957 18054 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 15:38:17.762929 18054 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786289732823074-3431-0/minicluster-data/master-0/data/instance:
uuid: "6f5372781f6b43b8a0ed721376c69069"
format_stamp: "Formatted at 2026-08-09 15:38:17 on dist-test-slave-2w3w"
server_key: "61091a8e85e3aad683b2e1de95fb84d8"
server_key_iv: "9aa2edcacb2b2991e12d659eb11edd39"
server_key_version: "encryptionkey@0"
I20260809 15:38:17.763516 18054 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786289732823074-3431-0/minicluster-data/master-0/wal/instance:
uuid: "6f5372781f6b43b8a0ed721376c69069"
format_stamp: "Formatted at 2026-08-09 15:38:17 on dist-test-slave-2w3w"
server_key: "61091a8e85e3aad683b2e1de95fb84d8"
server_key_iv: "9aa2edcacb2b2991e12d659eb11edd39"
server_key_version: "encryptionkey@0"
I20260809 15:38:17.767035 18054 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.002s sys 0.002s
I20260809 15:38:17.769433 18071 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 15:38:17.770632 18054 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260809 15:38:17.770787 18054 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786289732823074-3431-0/minicluster-data/master-0/data,/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786289732823074-3431-0/minicluster-data/master-0/wal
uuid: "6f5372781f6b43b8a0ed721376c69069"
format_stamp: "Formatted at 2026-08-09 15:38:17 on dist-test-slave-2w3w"
server_key: "61091a8e85e3aad683b2e1de95fb84d8"
server_key_iv: "9aa2edcacb2b2991e12d659eb11edd39"
server_key_version: "encryptionkey@0"
I20260809 15:38:17.770952 18054 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786289732823074-3431-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786289732823074-3431-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786289732823074-3431-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 15:38:17.792474 18054 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 15:38:17.795825 18054 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 15:38:17.796042 18054 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 15:38:17.803336 18054 rpc_server.cc:307] RPC server started. Bound to: 127.3.89.254:38743
I20260809 15:38:17.803359 18129 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.3.89.254:38743 every 8 connection(s)
I20260809 15:38:17.804334 18054 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786289732823074-3431-0/minicluster-data/master-0/data/info.pb
I20260809 15:38:17.807443 18130 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 15:38:17.812830 3431 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskm314YR/build/debug/bin/kudu as pid 18054
I20260809 15:38:17.812932 3431 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786289732823074-3431-0/minicluster-data/master-0/wal/instance
I20260809 15:38:17.813037 18130 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6f5372781f6b43b8a0ed721376c69069: Bootstrap starting.
I20260809 15:38:17.813284 3431 external_mini_cluster.cc:1428] Setting key 4b2330a4afc980fca998cbf4bfd1aef2
I20260809 15:38:17.815707 18130 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 6f5372781f6b43b8a0ed721376c69069: Neither blocks nor log segments found. Creating new log.
I20260809 15:38:17.816592 18130 log.cc:824] T 00000000000000000000000000000000 P 6f5372781f6b43b8a0ed721376c69069: Log is configured to *not* fsync() on all Append() calls
I20260809 15:38:17.818778 18130 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6f5372781f6b43b8a0ed721376c69069: No bootstrap required, opened a new log
I20260809 15:38:17.822736 18130 raft_consensus.cc:348] T 00000000000000000000000000000000 P 6f5372781f6b43b8a0ed721376c69069 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6f5372781f6b43b8a0ed721376c69069" member_type: VOTER last_known_addr { host: "127.3.89.254" port: 38743 } }
Aug 09 15:38:17 dist-test-slave-2w3w krb5kdc[18041](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786289897, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.3.89.254@KRBTEST.COM
I20260809 15:38:17.823207 18130 raft_consensus.cc:374] T 00000000000000000000000000000000 P 6f5372781f6b43b8a0ed721376c69069 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 15:38:17.823273 18130 raft_consensus.cc:729] T 00000000000000000000000000000000 P 6f5372781f6b43b8a0ed721376c69069 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6f5372781f6b43b8a0ed721376c69069, State: Initialized, Role: FOLLOWER
I20260809 15:38:17.823725 18130 consensus_queue.cc:260] T 00000000000000000000000000000000 P 6f5372781f6b43b8a0ed721376c69069 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6f5372781f6b43b8a0ed721376c69069" member_type: VOTER last_known_addr { host: "127.3.89.254" port: 38743 } }
I20260809 15:38:17.823899 18130 raft_consensus.cc:388] T 00000000000000000000000000000000 P 6f5372781f6b43b8a0ed721376c69069 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 15:38:17.823971 18130 raft_consensus.cc:482] T 00000000000000000000000000000000 P 6f5372781f6b43b8a0ed721376c69069 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 15:38:17.824074 18130 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 6f5372781f6b43b8a0ed721376c69069 [term 0 FOLLOWER]: Advancing to term 1
I20260809 15:38:17.825286 18130 raft_consensus.cc:504] T 00000000000000000000000000000000 P 6f5372781f6b43b8a0ed721376c69069 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6f5372781f6b43b8a0ed721376c69069" member_type: VOTER last_known_addr { host: "127.3.89.254" port: 38743 } }
I20260809 15:38:17.825646 18130 leader_election.cc:302] T 00000000000000000000000000000000 P 6f5372781f6b43b8a0ed721376c69069 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 6f5372781f6b43b8a0ed721376c69069; no voters:
I20260809 15:38:17.826001 18130 leader_election.cc:288] T 00000000000000000000000000000000 P 6f5372781f6b43b8a0ed721376c69069 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 15:38:17.826125 18135 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 6f5372781f6b43b8a0ed721376c69069 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 15:38:17.826362 18135 raft_consensus.cc:686] T 00000000000000000000000000000000 P 6f5372781f6b43b8a0ed721376c69069 [term 1 LEADER]: Becoming Leader. State: Replica: 6f5372781f6b43b8a0ed721376c69069, State: Running, Role: LEADER
I20260809 15:38:17.826758 18135 consensus_queue.cc:237] T 00000000000000000000000000000000 P 6f5372781f6b43b8a0ed721376c69069 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6f5372781f6b43b8a0ed721376c69069" member_type: VOTER last_known_addr { host: "127.3.89.254" port: 38743 } }
I20260809 15:38:17.827378 18130 sys_catalog.cc:564] T 00000000000000000000000000000000 P 6f5372781f6b43b8a0ed721376c69069 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 15:38:17.828954 18137 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6f5372781f6b43b8a0ed721376c69069 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 6f5372781f6b43b8a0ed721376c69069. Latest consensus state: current_term: 1 leader_uuid: "6f5372781f6b43b8a0ed721376c69069" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6f5372781f6b43b8a0ed721376c69069" member_type: VOTER last_known_addr { host: "127.3.89.254" port: 38743 } } }
I20260809 15:38:17.829360 18137 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6f5372781f6b43b8a0ed721376c69069 [sys.catalog]: This master's current role is: LEADER
I20260809 15:38:17.829463 18133 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:38:17.814646 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36784 (local address 127.3.89.254:38743)
0809 15:38:17.815141 (+ 495us) server_negotiation.cc:206] Beginning negotiation
0809 15:38:17.815159 (+ 18us) server_negotiation.cc:399] Waiting for connection header
0809 15:38:17.815201 (+ 42us) server_negotiation.cc:407] Connection header received
0809 15:38:17.816074 (+ 873us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 15:38:17.816096 (+ 22us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 15:38:17.816493 (+ 397us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 15:38:17.816839 (+ 346us) server_negotiation.cc:226] Negotiated authn=SASL
0809 15:38:17.817783 (+ 944us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:38:17.819254 (+ 1471us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:38:17.820104 (+ 850us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:38:17.820802 (+ 698us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:38:17.820987 (+ 185us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:38:17.823925 (+ 2938us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 15:38:17.823932 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 15:38:17.823948 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 15:38:17.823982 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0809 15:38:17.825958 (+ 1976us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 15:38:17.826533 (+ 575us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 15:38:17.826539 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 15:38:17.826544 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0809 15:38:17.826682 (+ 138us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 15:38:17.827045 (+ 363us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 15:38:17.827048 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 15:38:17.827050 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 15:38:17.827567 (+ 517us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 15:38:17.827758 (+ 191us) server_negotiation.cc:1035] Waiting for connection context
0809 15:38:17.828075 (+ 317us) server_negotiation.cc:299] Negotiation successful
0809 15:38:17.828426 (+ 351us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":352,"thread_start_us":124,"threads_started":1}
I20260809 15:38:17.829260 18136 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6f5372781f6b43b8a0ed721376c69069 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "6f5372781f6b43b8a0ed721376c69069" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6f5372781f6b43b8a0ed721376c69069" member_type: VOTER last_known_addr { host: "127.3.89.254" port: 38743 } } }
I20260809 15:38:17.831682 18136 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6f5372781f6b43b8a0ed721376c69069 [sys.catalog]: This master's current role is: LEADER
W20260809 15:38:17.832383 18149 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 6f5372781f6b43b8a0ed721376c69069: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260809 15:38:17.832468 18149 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260809 15:38:17.832690 18151 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 15:38:17.833047 3431 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260809 15:38:17.833556 18151 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 15:38:17.839242 18151 catalog_manager.cc:1347] Generated new cluster ID: fdd0c83a09184881b22cf7d0ccd51252
I20260809 15:38:17.839306 18151 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 15:38:17.861411 18151 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 15:38:17.862334 18151 catalog_manager.cc:1504] Loading token signing keys...
I20260809 15:38:17.873836 18151 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 6f5372781f6b43b8a0ed721376c69069: Generated new TSK 0
I20260809 15:38:17.874473 18151 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260809 15:38:17.889410 3431 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskm314YR/build/debug/bin/kudu with pid 18054
2026-08-09T15:38:17Z chronyd exiting
[ OK ] SecurityITest.IPKICACertWebServerEndPoints (1480 ms)
[ RUN ] SecurityITest.TestEnableAndDisableEncryptedIPKICert
Loading random data
Initializing database '/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786289732823074-3431-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 15:38:17 dist-test-slave-2w3w krb5kdc[18158](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 15:38:17 dist-test-slave-2w3w krb5kdc[18158](info): set up 2 sockets
Aug 09 15:38:17 dist-test-slave-2w3w krb5kdc[18158](info): commencing operation
krb5kdc: starting...
W20260809 15:38:18.968082 3431 mini_kdc.cc:121] Time spent starting KDC: real 1.056s user 0.001s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786289732823074-3431-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786289732823074-3431-0/krb5kdc/test-user.keytab.
Aug 09 15:38:19 dist-test-slave-2w3w krb5kdc[18158](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786289899, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T15:38:19Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T15:38:19Z Disabled control of system clock
WARNING: no policy specified for kudu/127.3.89.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.3.89.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.3.89.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786289732823074-3431-0/krb5kdc/kudu_127.3.89.254.keytab.
Entry for principal kudu/127.3.89.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786289732823074-3431-0/krb5kdc/kudu_127.3.89.254.keytab.
I20260809 15:38:19.094857 3431 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskm314YR/build/debug/bin/kudu
/tmp/dist-test-taskm314YR/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786289732823074-3431-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786289732823074-3431-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786289732823074-3431-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786289732823074-3431-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.3.89.254:33497
--webserver_interface=127.3.89.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786289732823074-3431-0/krb5kdc/kudu_127.3.89.254.keytab
--principal=kudu/127.3.89.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.3.89.212:43771
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.3.89.254:33497
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786289732823074-3431-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786289732823074-3431-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786289732823074-3431-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 15:38:19.202643 18172 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 15:38:19.202884 18172 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 15:38:19.202922 18172 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 15:38:19.206782 18172 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 15:38:19.206842 18172 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 15:38:19.206861 18172 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 15:38:19.206878 18172 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 15:38:19.206893 18172 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 15:38:19.212013 18172 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.3.89.212:43771
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786289732823074-3431-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786289732823074-3431-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.3.89.254:33497
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786289732823074-3431-0/krb5kdc/kudu_127.3.89.254.keytab
--principal=kudu/127.3.89.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.3.89.254:33497
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786289732823074-3431-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.3.89.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.18172
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786289732823074-3431-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 09 Aug 2026 14:46:55 UTC on ccycloud.quasar-yswkmf.root.comops.site
build id 4576540
I20260809 15:38:19.212838 18172 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 15:38:19.213706 18172 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 15:38:19.222044 18180 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 15:38:19.222080 18178 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 15:38:19.222129 18172 server_base.cc:1034] running on GCE node
W20260809 15:38:19.222035 18177 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 15:38:19.222796 18172 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 15:38:19.223845 18172 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 15:38:19.225031 18172 hybrid_clock.cc:648] HybridClock initialized: now 1786289899225004 us; error 39 us; skew 500 ppm
Aug 09 15:38:19 dist-test-slave-2w3w krb5kdc[18158](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786289899, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.89.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 15:38:19.227659 18172 init.cc:377] Logged in from keytab as kudu/127.3.89.254@KRBTEST.COM (short username kudu)
I20260809 15:38:19.228523 18172 webserver.cc:460] Webserver started at http://127.3.89.254:38433/ using document root <none> and password file <none>
I20260809 15:38:19.229056 18172 fs_manager.cc:359] Metadata directory not provided
I20260809 15:38:19.229149 18172 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 15:38:19.229369 18172 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 15:38:19.231352 18172 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786289732823074-3431-0/minicluster-data/master-0/data/instance:
uuid: "f5ecfca57914460d8be340f7b52004b5"
format_stamp: "Formatted at 2026-08-09 15:38:19 on dist-test-slave-2w3w"
server_key: "aaf2376a0c2aedc840940ad38cd4d160"
server_key_iv: "e61ba99cabf8f23507a5d3c83dc759f0"
server_key_version: "encryptionkey@0"
I20260809 15:38:19.231899 18172 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786289732823074-3431-0/minicluster-data/master-0/wal/instance:
uuid: "f5ecfca57914460d8be340f7b52004b5"
format_stamp: "Formatted at 2026-08-09 15:38:19 on dist-test-slave-2w3w"
server_key: "aaf2376a0c2aedc840940ad38cd4d160"
server_key_iv: "e61ba99cabf8f23507a5d3c83dc759f0"
server_key_version: "encryptionkey@0"
I20260809 15:38:19.235414 18172 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.003s sys 0.000s
I20260809 15:38:19.238006 18188 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 15:38:19.239491 18172 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260809 15:38:19.239619 18172 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786289732823074-3431-0/minicluster-data/master-0/data,/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786289732823074-3431-0/minicluster-data/master-0/wal
uuid: "f5ecfca57914460d8be340f7b52004b5"
format_stamp: "Formatted at 2026-08-09 15:38:19 on dist-test-slave-2w3w"
server_key: "aaf2376a0c2aedc840940ad38cd4d160"
server_key_iv: "e61ba99cabf8f23507a5d3c83dc759f0"
server_key_version: "encryptionkey@0"
I20260809 15:38:19.239732 18172 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786289732823074-3431-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786289732823074-3431-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786289732823074-3431-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 15:38:19.281751 18172 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 15:38:19.284700 18172 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 15:38:19.284914 18172 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 15:38:19.292093 18172 rpc_server.cc:307] RPC server started. Bound to: 127.3.89.254:33497
I20260809 15:38:19.292413 18241 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.3.89.254:33497 every 8 connection(s)
I20260809 15:38:19.293061 18172 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786289732823074-3431-0/minicluster-data/master-0/data/info.pb
I20260809 15:38:19.295765 18242 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 15:38:19.300979 18242 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f5ecfca57914460d8be340f7b52004b5: Bootstrap starting.
I20260809 15:38:19.301337 3431 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskm314YR/build/debug/bin/kudu as pid 18172
I20260809 15:38:19.301478 3431 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786289732823074-3431-0/minicluster-data/master-0/wal/instance
I20260809 15:38:19.301775 3431 external_mini_cluster.cc:1428] Setting key 80d81d402600c7e26abe20f9a6fefb4a
I20260809 15:38:19.303629 18242 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P f5ecfca57914460d8be340f7b52004b5: Neither blocks nor log segments found. Creating new log.
I20260809 15:38:19.304356 18242 log.cc:824] T 00000000000000000000000000000000 P f5ecfca57914460d8be340f7b52004b5: Log is configured to *not* fsync() on all Append() calls
I20260809 15:38:19.306493 18242 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f5ecfca57914460d8be340f7b52004b5: No bootstrap required, opened a new log
Aug 09 15:38:19 dist-test-slave-2w3w krb5kdc[18158](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786289899, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.3.89.254@KRBTEST.COM
I20260809 15:38:19.311064 18242 raft_consensus.cc:348] T 00000000000000000000000000000000 P f5ecfca57914460d8be340f7b52004b5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f5ecfca57914460d8be340f7b52004b5" member_type: VOTER last_known_addr { host: "127.3.89.254" port: 33497 } }
I20260809 15:38:19.311337 18242 raft_consensus.cc:374] T 00000000000000000000000000000000 P f5ecfca57914460d8be340f7b52004b5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 15:38:19.311432 18242 raft_consensus.cc:729] T 00000000000000000000000000000000 P f5ecfca57914460d8be340f7b52004b5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f5ecfca57914460d8be340f7b52004b5, State: Initialized, Role: FOLLOWER
I20260809 15:38:19.311915 18242 consensus_queue.cc:260] T 00000000000000000000000000000000 P f5ecfca57914460d8be340f7b52004b5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f5ecfca57914460d8be340f7b52004b5" member_type: VOTER last_known_addr { host: "127.3.89.254" port: 33497 } }
I20260809 15:38:19.312094 18242 raft_consensus.cc:388] T 00000000000000000000000000000000 P f5ecfca57914460d8be340f7b52004b5 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 15:38:19.312161 18242 raft_consensus.cc:482] T 00000000000000000000000000000000 P f5ecfca57914460d8be340f7b52004b5 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 15:38:19.312268 18242 raft_consensus.cc:3037] T 00000000000000000000000000000000 P f5ecfca57914460d8be340f7b52004b5 [term 0 FOLLOWER]: Advancing to term 1
I20260809 15:38:19.313426 18242 raft_consensus.cc:504] T 00000000000000000000000000000000 P f5ecfca57914460d8be340f7b52004b5 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f5ecfca57914460d8be340f7b52004b5" member_type: VOTER last_known_addr { host: "127.3.89.254" port: 33497 } }
I20260809 15:38:19.313803 18242 leader_election.cc:302] T 00000000000000000000000000000000 P f5ecfca57914460d8be340f7b52004b5 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: f5ecfca57914460d8be340f7b52004b5; no voters:
I20260809 15:38:19.314165 18242 leader_election.cc:288] T 00000000000000000000000000000000 P f5ecfca57914460d8be340f7b52004b5 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 15:38:19.314262 18248 raft_consensus.cc:2781] T 00000000000000000000000000000000 P f5ecfca57914460d8be340f7b52004b5 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 15:38:19.314492 18248 raft_consensus.cc:686] T 00000000000000000000000000000000 P f5ecfca57914460d8be340f7b52004b5 [term 1 LEADER]: Becoming Leader. State: Replica: f5ecfca57914460d8be340f7b52004b5, State: Running, Role: LEADER
I20260809 15:38:19.314834 18248 consensus_queue.cc:237] T 00000000000000000000000000000000 P f5ecfca57914460d8be340f7b52004b5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f5ecfca57914460d8be340f7b52004b5" member_type: VOTER last_known_addr { host: "127.3.89.254" port: 33497 } }
I20260809 15:38:19.315477 18242 sys_catalog.cc:564] T 00000000000000000000000000000000 P f5ecfca57914460d8be340f7b52004b5 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 15:38:19.317402 18246 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:38:19.303123 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47428 (local address 127.3.89.254:33497)
0809 15:38:19.303550 (+ 427us) server_negotiation.cc:206] Beginning negotiation
0809 15:38:19.303564 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0809 15:38:19.303598 (+ 34us) server_negotiation.cc:407] Connection header received
0809 15:38:19.304250 (+ 652us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 15:38:19.304266 (+ 16us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 15:38:19.304627 (+ 361us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 15:38:19.304994 (+ 367us) server_negotiation.cc:226] Negotiated authn=SASL
0809 15:38:19.305913 (+ 919us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:38:19.306949 (+ 1036us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:38:19.308189 (+ 1240us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:38:19.308646 (+ 457us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:38:19.308762 (+ 116us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:38:19.311706 (+ 2944us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 15:38:19.311712 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 15:38:19.311726 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 15:38:19.311756 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0809 15:38:19.313820 (+ 2064us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 15:38:19.314371 (+ 551us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 15:38:19.314377 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 15:38:19.314382 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0809 15:38:19.314549 (+ 167us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 15:38:19.314915 (+ 366us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 15:38:19.314918 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 15:38:19.314920 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 15:38:19.315442 (+ 522us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 15:38:19.315656 (+ 214us) server_negotiation.cc:1035] Waiting for connection context
0809 15:38:19.316045 (+ 389us) server_negotiation.cc:299] Negotiation successful
0809 15:38:19.316369 (+ 324us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":292,"thread_start_us":136,"threads_started":1}
I20260809 15:38:19.318138 18250 sys_catalog.cc:455] T 00000000000000000000000000000000 P f5ecfca57914460d8be340f7b52004b5 [sys.catalog]: SysCatalogTable state changed. Reason: New leader f5ecfca57914460d8be340f7b52004b5. Latest consensus state: current_term: 1 leader_uuid: "f5ecfca57914460d8be340f7b52004b5" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f5ecfca57914460d8be340f7b52004b5" member_type: VOTER last_known_addr { host: "127.3.89.254" port: 33497 } } }
I20260809 15:38:19.318255 18250 sys_catalog.cc:458] T 00000000000000000000000000000000 P f5ecfca57914460d8be340f7b52004b5 [sys.catalog]: This master's current role is: LEADER
I20260809 15:38:19.318501 18249 sys_catalog.cc:455] T 00000000000000000000000000000000 P f5ecfca57914460d8be340f7b52004b5 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "f5ecfca57914460d8be340f7b52004b5" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f5ecfca57914460d8be340f7b52004b5" member_type: VOTER last_known_addr { host: "127.3.89.254" port: 33497 } } }
I20260809 15:38:19.318603 18249 sys_catalog.cc:458] T 00000000000000000000000000000000 P f5ecfca57914460d8be340f7b52004b5 [sys.catalog]: This master's current role is: LEADER
W20260809 15:38:19.320466 18264 catalog_manager.cc:1558] T 00000000000000000000000000000000 P f5ecfca57914460d8be340f7b52004b5: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260809 15:38:19.320542 18264 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260809 15:38:19.320616 18265 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 15:38:19.321265 18265 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 15:38:19.325886 18265 catalog_manager.cc:1347] Generated new cluster ID: 6b5d21f311dd4f3e809e5ae566290b34
I20260809 15:38:19.325943 18265 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 15:38:19.338214 18265 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 15:38:19.339365 18265 catalog_manager.cc:1504] Loading token signing keys...
I20260809 15:38:19.347029 18265 catalog_manager.cc:5868] T 00000000000000000000000000000000 P f5ecfca57914460d8be340f7b52004b5: Generated new TSK 0
I20260809 15:38:19.347739 18265 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.3.89.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.3.89.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.3.89.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786289732823074-3431-0/krb5kdc/kudu_127.3.89.193.keytab.
Entry for principal kudu/127.3.89.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786289732823074-3431-0/krb5kdc/kudu_127.3.89.193.keytab.
I20260809 15:38:19.377558 3431 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskm314YR/build/debug/bin/kudu
/tmp/dist-test-taskm314YR/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786289732823074-3431-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786289732823074-3431-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786289732823074-3431-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786289732823074-3431-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.3.89.193:0
--local_ip_for_outbound_sockets=127.3.89.193
--webserver_interface=127.3.89.193
--webserver_port=0
--tserver_master_addrs=127.3.89.254:33497
--keytab_file=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786289732823074-3431-0/krb5kdc/kudu_127.3.89.193.keytab
--principal=kudu/127.3.89.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.3.89.212:43771
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786289732823074-3431-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786289732823074-3431-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786289732823074-3431-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 15:38:19.487721 18271 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 15:38:19.487962 18271 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 15:38:19.488003 18271 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 15:38:19.491935 18271 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 15:38:19.491982 18271 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 15:38:19.492036 18271 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.3.89.193
I20260809 15:38:19.497041 18271 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.3.89.212:43771
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786289732823074-3431-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786289732823074-3431-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786289732823074-3431-0/krb5kdc/kudu_127.3.89.193.keytab
--principal=kudu/127.3.89.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.3.89.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786289732823074-3431-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.3.89.193
--webserver_port=0
--tserver_master_addrs=127.3.89.254:33497
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.18271
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.3.89.193
--log_dir=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786289732823074-3431-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 09 Aug 2026 14:46:55 UTC on ccycloud.quasar-yswkmf.root.comops.site
build id 4576540
I20260809 15:38:19.497902 18271 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 15:38:19.498775 18271 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 15:38:19.507911 18276 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 15:38:19.507933 18279 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 15:38:19.507979 18277 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 15:38:19.508285 18271 server_base.cc:1034] running on GCE node
I20260809 15:38:19.508664 18271 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 15:38:19.509282 18271 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 15:38:19.510450 18271 hybrid_clock.cc:648] HybridClock initialized: now 1786289899510430 us; error 30 us; skew 500 ppm
Aug 09 15:38:19 dist-test-slave-2w3w krb5kdc[18158](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786289899, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.89.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 15:38:19.518543 18271 init.cc:377] Logged in from keytab as kudu/127.3.89.193@KRBTEST.COM (short username kudu)
I20260809 15:38:19.519402 18271 webserver.cc:460] Webserver started at http://127.3.89.193:42811/ using document root <none> and password file <none>
I20260809 15:38:19.519919 18271 fs_manager.cc:359] Metadata directory not provided
I20260809 15:38:19.519979 18271 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 15:38:19.520195 18271 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 15:38:19.522145 18271 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786289732823074-3431-0/minicluster-data/ts-0/data/instance:
uuid: "b0a56d93c098468296a2000c9c5609ca"
format_stamp: "Formatted at 2026-08-09 15:38:19 on dist-test-slave-2w3w"
server_key: "f0ee155f811f0aa13c8b3b8f3e49f2bb"
server_key_iv: "5e2a5a61d21e9b6c20da06f34bb7cbf2"
server_key_version: "encryptionkey@0"
I20260809 15:38:19.522691 18271 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786289732823074-3431-0/minicluster-data/ts-0/wal/instance:
uuid: "b0a56d93c098468296a2000c9c5609ca"
format_stamp: "Formatted at 2026-08-09 15:38:19 on dist-test-slave-2w3w"
server_key: "f0ee155f811f0aa13c8b3b8f3e49f2bb"
server_key_iv: "5e2a5a61d21e9b6c20da06f34bb7cbf2"
server_key_version: "encryptionkey@0"
I20260809 15:38:19.526494 18271 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.006s sys 0.001s
I20260809 15:38:19.528724 18287 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 15:38:19.529965 18271 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.000s sys 0.002s
I20260809 15:38:19.530130 18271 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786289732823074-3431-0/minicluster-data/ts-0/data,/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786289732823074-3431-0/minicluster-data/ts-0/wal
uuid: "b0a56d93c098468296a2000c9c5609ca"
format_stamp: "Formatted at 2026-08-09 15:38:19 on dist-test-slave-2w3w"
server_key: "f0ee155f811f0aa13c8b3b8f3e49f2bb"
server_key_iv: "5e2a5a61d21e9b6c20da06f34bb7cbf2"
server_key_version: "encryptionkey@0"
I20260809 15:38:19.530262 18271 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786289732823074-3431-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786289732823074-3431-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786289732823074-3431-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 15:38:19.541071 18271 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 15:38:19.544231 18271 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 15:38:19.544413 18271 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 15:38:19.544926 18271 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 15:38:19.545672 18271 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 15:38:19.545749 18271 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 15:38:19.545812 18271 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 15:38:19.545858 18271 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 15:38:19.554450 18271 rpc_server.cc:307] RPC server started. Bound to: 127.3.89.193:44707
I20260809 15:38:19.554484 18409 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.3.89.193:44707 every 8 connection(s)
I20260809 15:38:19.555354 18271 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786289732823074-3431-0/minicluster-data/ts-0/data/info.pb
I20260809 15:38:19.563906 3431 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskm314YR/build/debug/bin/kudu as pid 18271
I20260809 15:38:19.564049 3431 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786289732823074-3431-0/minicluster-data/ts-0/wal/instance
I20260809 15:38:19.564337 3431 external_mini_cluster.cc:1428] Setting key dac43f75ab35208b16a111a51463d891
Aug 09 15:38:19 dist-test-slave-2w3w krb5kdc[18158](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786289899, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.89.193@KRBTEST.COM for kudu/127.3.89.254@KRBTEST.COM
I20260809 15:38:19.569049 18246 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:38:19.557075 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.89.193:33231 (local address 127.3.89.254:33497)
0809 15:38:19.557217 (+ 142us) server_negotiation.cc:206] Beginning negotiation
0809 15:38:19.557222 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 15:38:19.558062 (+ 840us) server_negotiation.cc:407] Connection header received
0809 15:38:19.559560 (+ 1498us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 15:38:19.559563 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 15:38:19.559607 (+ 44us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 15:38:19.559681 (+ 74us) server_negotiation.cc:226] Negotiated authn=SASL
0809 15:38:19.560745 (+ 1064us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:38:19.561311 (+ 566us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:38:19.562426 (+ 1115us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:38:19.562657 (+ 231us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:38:19.562735 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:38:19.565317 (+ 2582us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 15:38:19.565321 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 15:38:19.565322 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 15:38:19.565347 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0809 15:38:19.566763 (+ 1416us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 15:38:19.567456 (+ 693us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 15:38:19.567459 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 15:38:19.567460 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 15:38:19.567539 (+ 79us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 15:38:19.567949 (+ 410us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 15:38:19.567952 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 15:38:19.567953 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 15:38:19.568127 (+ 174us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 15:38:19.568243 (+ 116us) server_negotiation.cc:1035] Waiting for connection context
0809 15:38:19.568726 (+ 483us) server_negotiation.cc:299] Negotiation successful
0809 15:38:19.568887 (+ 161us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":61}
I20260809 15:38:19.569974 18412 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:38:19.557387 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.89.254:33497 (local address 127.3.89.193:33231)
0809 15:38:19.557865 (+ 478us) negotiation.cc:107] Waiting for socket to connect
0809 15:38:19.557906 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0809 15:38:19.559373 (+ 1467us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 15:38:19.559854 (+ 481us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 15:38:19.559860 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 15:38:19.560187 (+ 327us) client_negotiation.cc:189] Negotiated authn=SASL
0809 15:38:19.560609 (+ 422us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:38:19.560621 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:38:19.561450 (+ 829us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:38:19.561453 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:38:19.562303 (+ 850us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:38:19.562313 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:38:19.562817 (+ 504us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:38:19.562820 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:38:19.562952 (+ 132us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:38:19.563524 (+ 572us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 15:38:19.563539 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0809 15:38:19.565135 (+ 1596us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 15:38:19.566925 (+ 1790us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 15:38:19.566934 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 15:38:19.566950 (+ 16us) client_negotiation.cc:760] calling sasl_client_step()
0809 15:38:19.567344 (+ 394us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 15:38:19.567633 (+ 289us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 15:38:19.567639 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 15:38:19.567644 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0809 15:38:19.567831 (+ 187us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 15:38:19.568244 (+ 413us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 15:38:19.568251 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 15:38:19.568485 (+ 234us) client_negotiation.cc:769] Sending connection context
0809 15:38:19.568666 (+ 181us) client_negotiation.cc:240] Negotiation successful
0809 15:38:19.568972 (+ 306us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":268,"thread_start_us":114,"threads_started":1}
I20260809 15:38:19.571300 18410 heartbeater.cc:347] Connected to a master server at 127.3.89.254:33497
I20260809 15:38:19.571537 18410 heartbeater.cc:464] Registering TS with master...
I20260809 15:38:19.571990 18410 heartbeater.cc:511] Master 127.3.89.254:33497 requested a full tablet report, sending...
I20260809 15:38:19.573438 18205 ts_manager.cc:194] Registered new tserver with Master: b0a56d93c098468296a2000c9c5609ca (127.3.89.193:44707)
I20260809 15:38:19.574563 18205 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.3.89.193@KRBTEST.COM'} at 127.3.89.193:33231
WARNING: no policy specified for kudu/127.3.89.194@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.3.89.194@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.3.89.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786289732823074-3431-0/krb5kdc/kudu_127.3.89.194.keytab.
Entry for principal kudu/127.3.89.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786289732823074-3431-0/krb5kdc/kudu_127.3.89.194.keytab.
I20260809 15:38:19.594445 3431 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskm314YR/build/debug/bin/kudu
/tmp/dist-test-taskm314YR/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786289732823074-3431-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786289732823074-3431-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786289732823074-3431-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786289732823074-3431-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.3.89.194:0
--local_ip_for_outbound_sockets=127.3.89.194
--webserver_interface=127.3.89.194
--webserver_port=0
--tserver_master_addrs=127.3.89.254:33497
--keytab_file=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786289732823074-3431-0/krb5kdc/kudu_127.3.89.194.keytab
--principal=kudu/127.3.89.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.3.89.212:43771
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786289732823074-3431-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786289732823074-3431-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786289732823074-3431-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 15:38:19.702452 18416 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 15:38:19.702754 18416 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 15:38:19.702833 18416 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 15:38:19.706662 18416 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 15:38:19.706784 18416 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 15:38:19.706892 18416 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.3.89.194
I20260809 15:38:19.711931 18416 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.3.89.212:43771
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786289732823074-3431-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786289732823074-3431-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786289732823074-3431-0/krb5kdc/kudu_127.3.89.194.keytab
--principal=kudu/127.3.89.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.3.89.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786289732823074-3431-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.3.89.194
--webserver_port=0
--tserver_master_addrs=127.3.89.254:33497
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.18416
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.3.89.194
--log_dir=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786289732823074-3431-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 09 Aug 2026 14:46:55 UTC on ccycloud.quasar-yswkmf.root.comops.site
build id 4576540
I20260809 15:38:19.712888 18416 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 15:38:19.713796 18416 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 15:38:19.723203 18421 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 15:38:19.723284 18416 server_base.cc:1034] running on GCE node
W20260809 15:38:19.723227 18424 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 15:38:19.723232 18422 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 15:38:19.723834 18416 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 15:38:19.724356 18416 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 15:38:19.725517 18416 hybrid_clock.cc:648] HybridClock initialized: now 1786289899725499 us; error 30 us; skew 500 ppm
Aug 09 15:38:19 dist-test-slave-2w3w krb5kdc[18158](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786289899, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.89.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 15:38:19.728220 18416 init.cc:377] Logged in from keytab as kudu/127.3.89.194@KRBTEST.COM (short username kudu)
I20260809 15:38:19.729074 18416 webserver.cc:460] Webserver started at http://127.3.89.194:40495/ using document root <none> and password file <none>
I20260809 15:38:19.729619 18416 fs_manager.cc:359] Metadata directory not provided
I20260809 15:38:19.729722 18416 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 15:38:19.729988 18416 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 15:38:19.731993 18416 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786289732823074-3431-0/minicluster-data/ts-1/data/instance:
uuid: "273a5fb47ef740c196b1a55c87ed7466"
format_stamp: "Formatted at 2026-08-09 15:38:19 on dist-test-slave-2w3w"
server_key: "c6c967900852bcc2f6e79e6f877d02d2"
server_key_iv: "9d091753c3ded8f1cc2acd366d1e459d"
server_key_version: "encryptionkey@0"
I20260809 15:38:19.732563 18416 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786289732823074-3431-0/minicluster-data/ts-1/wal/instance:
uuid: "273a5fb47ef740c196b1a55c87ed7466"
format_stamp: "Formatted at 2026-08-09 15:38:19 on dist-test-slave-2w3w"
server_key: "c6c967900852bcc2f6e79e6f877d02d2"
server_key_iv: "9d091753c3ded8f1cc2acd366d1e459d"
server_key_version: "encryptionkey@0"
I20260809 15:38:19.736218 18416 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.004s sys 0.000s
I20260809 15:38:19.738480 18431 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 15:38:19.739666 18416 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260809 15:38:19.739845 18416 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786289732823074-3431-0/minicluster-data/ts-1/data,/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786289732823074-3431-0/minicluster-data/ts-1/wal
uuid: "273a5fb47ef740c196b1a55c87ed7466"
format_stamp: "Formatted at 2026-08-09 15:38:19 on dist-test-slave-2w3w"
server_key: "c6c967900852bcc2f6e79e6f877d02d2"
server_key_iv: "9d091753c3ded8f1cc2acd366d1e459d"
server_key_version: "encryptionkey@0"
I20260809 15:38:19.739955 18416 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786289732823074-3431-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786289732823074-3431-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786289732823074-3431-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 15:38:19.777958 18416 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 15:38:19.780903 18416 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 15:38:19.781113 18416 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 15:38:19.781703 18416 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 15:38:19.782523 18416 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 15:38:19.782596 18416 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 15:38:19.782665 18416 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 15:38:19.782716 18416 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 15:38:19.791872 18416 rpc_server.cc:307] RPC server started. Bound to: 127.3.89.194:45639
I20260809 15:38:19.791909 18553 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.3.89.194:45639 every 8 connection(s)
I20260809 15:38:19.792708 18416 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786289732823074-3431-0/minicluster-data/ts-1/data/info.pb
Aug 09 15:38:19 dist-test-slave-2w3w krb5kdc[18158](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786289899, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.89.194@KRBTEST.COM for kudu/127.3.89.254@KRBTEST.COM
I20260809 15:38:19.801594 3431 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskm314YR/build/debug/bin/kudu as pid 18416
I20260809 15:38:19.801716 3431 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786289732823074-3431-0/minicluster-data/ts-1/wal/instance
I20260809 15:38:19.802037 3431 external_mini_cluster.cc:1428] Setting key ece34dba227896e8dccdb445ad5728f8
I20260809 15:38:19.805774 18246 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:38:19.794487 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.89.194:58129 (local address 127.3.89.254:33497)
0809 15:38:19.794624 (+ 137us) server_negotiation.cc:206] Beginning negotiation
0809 15:38:19.794627 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0809 15:38:19.795434 (+ 807us) server_negotiation.cc:407] Connection header received
0809 15:38:19.796604 (+ 1170us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 15:38:19.796607 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 15:38:19.796654 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 15:38:19.796728 (+ 74us) server_negotiation.cc:226] Negotiated authn=SASL
0809 15:38:19.797773 (+ 1045us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:38:19.798301 (+ 528us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:38:19.799436 (+ 1135us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:38:19.799643 (+ 207us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:38:19.799721 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:38:19.802185 (+ 2464us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 15:38:19.802189 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 15:38:19.802196 (+ 7us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 15:38:19.802220 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0809 15:38:19.803779 (+ 1559us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 15:38:19.804296 (+ 517us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 15:38:19.804299 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 15:38:19.804305 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0809 15:38:19.804377 (+ 72us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 15:38:19.804719 (+ 342us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 15:38:19.804723 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 15:38:19.804725 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 15:38:19.804896 (+ 171us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 15:38:19.805012 (+ 116us) server_negotiation.cc:1035] Waiting for connection context
0809 15:38:19.805496 (+ 484us) server_negotiation.cc:299] Negotiation successful
0809 15:38:19.805613 (+ 117us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":51}
I20260809 15:38:19.806493 18556 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:38:19.794837 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.89.254:33497 (local address 127.3.89.194:58129)
0809 15:38:19.795271 (+ 434us) negotiation.cc:107] Waiting for socket to connect
0809 15:38:19.795314 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0809 15:38:19.796402 (+ 1088us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 15:38:19.796875 (+ 473us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 15:38:19.796880 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 15:38:19.797201 (+ 321us) client_negotiation.cc:189] Negotiated authn=SASL
0809 15:38:19.797645 (+ 444us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:38:19.797658 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:38:19.798408 (+ 750us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:38:19.798411 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:38:19.799335 (+ 924us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:38:19.799345 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:38:19.799788 (+ 443us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:38:19.799791 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:38:19.799917 (+ 126us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:38:19.800366 (+ 449us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 15:38:19.800380 (+ 14us) client_negotiation.cc:656] calling sasl_client_start()
0809 15:38:19.801937 (+ 1557us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 15:38:19.803901 (+ 1964us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 15:38:19.803906 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 15:38:19.803918 (+ 12us) client_negotiation.cc:760] calling sasl_client_step()
0809 15:38:19.804167 (+ 249us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 15:38:19.804473 (+ 306us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 15:38:19.804476 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 15:38:19.804478 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 15:38:19.804604 (+ 126us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 15:38:19.805021 (+ 417us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 15:38:19.805028 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 15:38:19.805266 (+ 238us) client_negotiation.cc:769] Sending connection context
0809 15:38:19.805446 (+ 180us) client_negotiation.cc:240] Negotiation successful
0809 15:38:19.805741 (+ 295us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":262,"thread_start_us":126,"threads_started":1}
I20260809 15:38:19.807741 18554 heartbeater.cc:347] Connected to a master server at 127.3.89.254:33497
I20260809 15:38:19.807974 18554 heartbeater.cc:464] Registering TS with master...
I20260809 15:38:19.808420 18554 heartbeater.cc:511] Master 127.3.89.254:33497 requested a full tablet report, sending...
I20260809 15:38:19.809525 18205 ts_manager.cc:194] Registered new tserver with Master: 273a5fb47ef740c196b1a55c87ed7466 (127.3.89.194:45639)
I20260809 15:38:19.810105 18205 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.3.89.194@KRBTEST.COM'} at 127.3.89.194:58129
WARNING: no policy specified for kudu/127.3.89.195@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.3.89.195@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.3.89.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786289732823074-3431-0/krb5kdc/kudu_127.3.89.195.keytab.
Entry for principal kudu/127.3.89.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786289732823074-3431-0/krb5kdc/kudu_127.3.89.195.keytab.
I20260809 15:38:19.833537 3431 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskm314YR/build/debug/bin/kudu
/tmp/dist-test-taskm314YR/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786289732823074-3431-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786289732823074-3431-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786289732823074-3431-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786289732823074-3431-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.3.89.195:0
--local_ip_for_outbound_sockets=127.3.89.195
--webserver_interface=127.3.89.195
--webserver_port=0
--tserver_master_addrs=127.3.89.254:33497
--keytab_file=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786289732823074-3431-0/krb5kdc/kudu_127.3.89.195.keytab
--principal=kudu/127.3.89.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.3.89.212:43771
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786289732823074-3431-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786289732823074-3431-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786289732823074-3431-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 15:38:19.941401 18559 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 15:38:19.941633 18559 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 15:38:19.941675 18559 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 15:38:19.945749 18559 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 15:38:19.945804 18559 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 15:38:19.945863 18559 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.3.89.195
I20260809 15:38:19.950891 18559 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.3.89.212:43771
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786289732823074-3431-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786289732823074-3431-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786289732823074-3431-0/krb5kdc/kudu_127.3.89.195.keytab
--principal=kudu/127.3.89.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.3.89.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786289732823074-3431-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.3.89.195
--webserver_port=0
--tserver_master_addrs=127.3.89.254:33497
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.18559
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.3.89.195
--log_dir=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786289732823074-3431-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 09 Aug 2026 14:46:55 UTC on ccycloud.quasar-yswkmf.root.comops.site
build id 4576540
I20260809 15:38:19.951886 18559 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 15:38:19.952806 18559 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 15:38:19.962162 18564 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 15:38:19.962117 18568 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 15:38:19.962129 18559 server_base.cc:1034] running on GCE node
W20260809 15:38:19.962148 18565 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 15:38:19.962792 18559 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 15:38:19.963416 18559 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 15:38:19.964579 18559 hybrid_clock.cc:648] HybridClock initialized: now 1786289899964567 us; error 24 us; skew 500 ppm
Aug 09 15:38:19 dist-test-slave-2w3w krb5kdc[18158](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786289899, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.89.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 15:38:19.967239 18559 init.cc:377] Logged in from keytab as kudu/127.3.89.195@KRBTEST.COM (short username kudu)
I20260809 15:38:19.968045 18559 webserver.cc:460] Webserver started at http://127.3.89.195:32809/ using document root <none> and password file <none>
I20260809 15:38:19.968591 18559 fs_manager.cc:359] Metadata directory not provided
I20260809 15:38:19.968688 18559 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 15:38:19.968917 18559 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 15:38:19.970940 18559 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786289732823074-3431-0/minicluster-data/ts-2/data/instance:
uuid: "421cb337efc747e292ba53a026c44c02"
format_stamp: "Formatted at 2026-08-09 15:38:19 on dist-test-slave-2w3w"
server_key: "eb2f06f9d711076fc80f282898368483"
server_key_iv: "b0314ff48b906d23f3b0478ecc39a789"
server_key_version: "encryptionkey@0"
I20260809 15:38:19.971561 18559 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786289732823074-3431-0/minicluster-data/ts-2/wal/instance:
uuid: "421cb337efc747e292ba53a026c44c02"
format_stamp: "Formatted at 2026-08-09 15:38:19 on dist-test-slave-2w3w"
server_key: "eb2f06f9d711076fc80f282898368483"
server_key_iv: "b0314ff48b906d23f3b0478ecc39a789"
server_key_version: "encryptionkey@0"
I20260809 15:38:19.975607 18559 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.001s
I20260809 15:38:19.977846 18575 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 15:38:19.979066 18559 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260809 15:38:19.979194 18559 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786289732823074-3431-0/minicluster-data/ts-2/data,/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786289732823074-3431-0/minicluster-data/ts-2/wal
uuid: "421cb337efc747e292ba53a026c44c02"
format_stamp: "Formatted at 2026-08-09 15:38:19 on dist-test-slave-2w3w"
server_key: "eb2f06f9d711076fc80f282898368483"
server_key_iv: "b0314ff48b906d23f3b0478ecc39a789"
server_key_version: "encryptionkey@0"
I20260809 15:38:19.979290 18559 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786289732823074-3431-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786289732823074-3431-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786289732823074-3431-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 15:38:19.993103 18559 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 15:38:19.995792 18559 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 15:38:19.996001 18559 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 15:38:19.996533 18559 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 15:38:19.997318 18559 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 15:38:19.997392 18559 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 15:38:19.997457 18559 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 15:38:19.997511 18559 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 15:38:20.006057 18559 rpc_server.cc:307] RPC server started. Bound to: 127.3.89.195:40257
I20260809 15:38:20.006089 18692 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.3.89.195:40257 every 8 connection(s)
I20260809 15:38:20.006888 18559 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786289732823074-3431-0/minicluster-data/ts-2/data/info.pb
I20260809 15:38:20.009994 3431 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskm314YR/build/debug/bin/kudu as pid 18559
I20260809 15:38:20.010155 3431 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786289732823074-3431-0/minicluster-data/ts-2/wal/instance
I20260809 15:38:20.010519 3431 external_mini_cluster.cc:1428] Setting key c1052cd3fd3b2d45e2250202b21caea9
Aug 09 15:38:20 dist-test-slave-2w3w krb5kdc[18158](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786289899, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.89.195@KRBTEST.COM for kudu/127.3.89.254@KRBTEST.COM
I20260809 15:38:20.020869 18246 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:38:20.008675 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.89.195:55677 (local address 127.3.89.254:33497)
0809 15:38:20.008856 (+ 181us) server_negotiation.cc:206] Beginning negotiation
0809 15:38:20.008861 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 15:38:20.009621 (+ 760us) server_negotiation.cc:407] Connection header received
0809 15:38:20.011258 (+ 1637us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 15:38:20.011261 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 15:38:20.011317 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 15:38:20.011408 (+ 91us) server_negotiation.cc:226] Negotiated authn=SASL
0809 15:38:20.012878 (+ 1470us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:38:20.013403 (+ 525us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:38:20.014481 (+ 1078us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:38:20.014684 (+ 203us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:38:20.014777 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:38:20.017473 (+ 2696us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 15:38:20.017476 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 15:38:20.017477 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 15:38:20.017499 (+ 22us) server_negotiation.cc:922] calling sasl_server_start()
0809 15:38:20.018770 (+ 1271us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 15:38:20.019316 (+ 546us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 15:38:20.019320 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 15:38:20.019322 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 15:38:20.019428 (+ 106us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 15:38:20.019807 (+ 379us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 15:38:20.019810 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 15:38:20.019811 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 15:38:20.020006 (+ 195us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 15:38:20.020120 (+ 114us) server_negotiation.cc:1035] Waiting for connection context
0809 15:38:20.020583 (+ 463us) server_negotiation.cc:299] Negotiation successful
0809 15:38:20.020703 (+ 120us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":72}
I20260809 15:38:20.021564 18696 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:38:20.009048 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.89.254:33497 (local address 127.3.89.195:55677)
0809 15:38:20.009465 (+ 417us) negotiation.cc:107] Waiting for socket to connect
0809 15:38:20.009505 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0809 15:38:20.011016 (+ 1511us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 15:38:20.011620 (+ 604us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 15:38:20.011629 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 15:38:20.012168 (+ 539us) client_negotiation.cc:189] Negotiated authn=SASL
0809 15:38:20.012714 (+ 546us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:38:20.012731 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:38:20.013512 (+ 781us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:38:20.013515 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:38:20.014373 (+ 858us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:38:20.014382 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:38:20.014816 (+ 434us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:38:20.014819 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:38:20.015055 (+ 236us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:38:20.015688 (+ 633us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 15:38:20.015704 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0809 15:38:20.017331 (+ 1627us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 15:38:20.018883 (+ 1552us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 15:38:20.018887 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 15:38:20.018897 (+ 10us) client_negotiation.cc:760] calling sasl_client_step()
0809 15:38:20.019200 (+ 303us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 15:38:20.019557 (+ 357us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 15:38:20.019562 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 15:38:20.019564 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 15:38:20.019695 (+ 131us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 15:38:20.020117 (+ 422us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 15:38:20.020123 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 15:38:20.020333 (+ 210us) client_negotiation.cc:769] Sending connection context
0809 15:38:20.020524 (+ 191us) client_negotiation.cc:240] Negotiation successful
0809 15:38:20.020816 (+ 292us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":247,"thread_start_us":109,"threads_started":1}
I20260809 15:38:20.022601 18694 heartbeater.cc:347] Connected to a master server at 127.3.89.254:33497
I20260809 15:38:20.022796 18694 heartbeater.cc:464] Registering TS with master...
I20260809 15:38:20.023384 18694 heartbeater.cc:511] Master 127.3.89.254:33497 requested a full tablet report, sending...
I20260809 15:38:20.024430 18204 ts_manager.cc:194] Registered new tserver with Master: 421cb337efc747e292ba53a026c44c02 (127.3.89.195:40257)
I20260809 15:38:20.025004 18204 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.3.89.195@KRBTEST.COM'} at 127.3.89.195:55677
I20260809 15:38:20.036127 3431 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260809 15:38:20.045378 18246 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:38:20.038298 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47444 (local address 127.3.89.254:33497)
0809 15:38:20.038408 (+ 110us) server_negotiation.cc:206] Beginning negotiation
0809 15:38:20.038412 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 15:38:20.038599 (+ 187us) server_negotiation.cc:407] Connection header received
0809 15:38:20.038692 (+ 93us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 15:38:20.038694 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 15:38:20.038729 (+ 35us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 15:38:20.038793 (+ 64us) server_negotiation.cc:226] Negotiated authn=SASL
0809 15:38:20.039483 (+ 690us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:38:20.039965 (+ 482us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:38:20.040749 (+ 784us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:38:20.040951 (+ 202us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:38:20.041046 (+ 95us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:38:20.042253 (+ 1207us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 15:38:20.042255 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 15:38:20.042257 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 15:38:20.042278 (+ 21us) server_negotiation.cc:922] calling sasl_server_start()
0809 15:38:20.043922 (+ 1644us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 15:38:20.044347 (+ 425us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 15:38:20.044349 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 15:38:20.044350 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 15:38:20.044479 (+ 129us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 15:38:20.044711 (+ 232us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 15:38:20.044713 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 15:38:20.044714 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 15:38:20.044905 (+ 191us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 15:38:20.045007 (+ 102us) server_negotiation.cc:1035] Waiting for connection context
0809 15:38:20.045187 (+ 180us) server_negotiation.cc:299] Negotiation successful
0809 15:38:20.045271 (+ 84us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":44}
I20260809 15:38:20.048218 18204 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:47444:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260809 15:38:20.051206 18204 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260809 15:38:20.065315 18707 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:38:20.059260 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.89.195:40257 (local address 127.0.0.1:33674)
0809 15:38:20.059803 (+ 543us) negotiation.cc:107] Waiting for socket to connect
0809 15:38:20.059878 (+ 75us) client_negotiation.cc:174] Beginning negotiation
0809 15:38:20.060010 (+ 132us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 15:38:20.061101 (+ 1091us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 15:38:20.061106 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 15:38:20.061127 (+ 21us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 15:38:20.061266 (+ 139us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:38:20.061273 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:38:20.062356 (+ 1083us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:38:20.062362 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:38:20.063605 (+ 1243us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:38:20.063616 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:38:20.064759 (+ 1143us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:38:20.064763 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:38:20.064872 (+ 109us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:38:20.064919 (+ 47us) client_negotiation.cc:769] Sending connection context
0809 15:38:20.065029 (+ 110us) client_negotiation.cc:240] Negotiation successful
0809 15:38:20.065141 (+ 112us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":382,"thread_start_us":85,"threads_started":1}
I20260809 15:38:20.065352 18706 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:38:20.059429 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33674 (local address 127.3.89.195:40257)
0809 15:38:20.060597 (+ 1168us) server_negotiation.cc:206] Beginning negotiation
0809 15:38:20.060607 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0809 15:38:20.060652 (+ 45us) server_negotiation.cc:407] Connection header received
0809 15:38:20.060733 (+ 81us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 15:38:20.060738 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 15:38:20.060940 (+ 202us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 15:38:20.061107 (+ 167us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 15:38:20.061393 (+ 286us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:38:20.062217 (+ 824us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:38:20.063748 (+ 1531us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:38:20.064624 (+ 876us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:38:20.064739 (+ 115us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:38:20.064896 (+ 157us) server_negotiation.cc:1035] Waiting for connection context
0809 15:38:20.065080 (+ 184us) server_negotiation.cc:299] Negotiation successful
0809 15:38:20.065215 (+ 135us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1053,"thread_start_us":117,"threads_started":1}
I20260809 15:38:20.065631 18705 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:38:20.059472 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.89.193:44707 (local address 127.0.0.1:56864)
0809 15:38:20.060238 (+ 766us) negotiation.cc:107] Waiting for socket to connect
0809 15:38:20.060251 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0809 15:38:20.060330 (+ 79us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 15:38:20.061267 (+ 937us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 15:38:20.061270 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 15:38:20.061340 (+ 70us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 15:38:20.061459 (+ 119us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:38:20.061464 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:38:20.063073 (+ 1609us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:38:20.063079 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:38:20.064181 (+ 1102us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:38:20.064191 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:38:20.065242 (+ 1051us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:38:20.065245 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:38:20.065341 (+ 96us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:38:20.065373 (+ 32us) client_negotiation.cc:769] Sending connection context
0809 15:38:20.065431 (+ 58us) client_negotiation.cc:240] Negotiation successful
0809 15:38:20.065502 (+ 71us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":700,"mutex_wait_us":37,"spinlock_wait_cycles":8704,"thread_start_us":92,"threads_started":1}
I20260809 15:38:20.066092 18709 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:38:20.060177 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.89.194:45639 (local address 127.0.0.1:37828)
0809 15:38:20.060399 (+ 222us) negotiation.cc:107] Waiting for socket to connect
0809 15:38:20.060413 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0809 15:38:20.060488 (+ 75us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 15:38:20.061279 (+ 791us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 15:38:20.061282 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 15:38:20.061299 (+ 17us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 15:38:20.061436 (+ 137us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:38:20.061446 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:38:20.062912 (+ 1466us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:38:20.062915 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:38:20.064466 (+ 1551us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:38:20.064471 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:38:20.065719 (+ 1248us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:38:20.065723 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:38:20.065840 (+ 117us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:38:20.065861 (+ 21us) client_negotiation.cc:769] Sending connection context
0809 15:38:20.065929 (+ 68us) client_negotiation.cc:240] Negotiation successful
0809 15:38:20.065998 (+ 69us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":158,"mutex_wait_us":83,"thread_start_us":108,"threads_started":1}
I20260809 15:38:20.066361 18708 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:38:20.060004 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56864 (local address 127.3.89.193:44707)
0809 15:38:20.060795 (+ 791us) server_negotiation.cc:206] Beginning negotiation
0809 15:38:20.060810 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0809 15:38:20.060828 (+ 18us) server_negotiation.cc:407] Connection header received
0809 15:38:20.060965 (+ 137us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 15:38:20.060969 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 15:38:20.061121 (+ 152us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 15:38:20.061655 (+ 534us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 15:38:20.061806 (+ 151us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:38:20.062880 (+ 1074us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:38:20.064286 (+ 1406us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:38:20.065090 (+ 804us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:38:20.065822 (+ 732us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:38:20.065956 (+ 134us) server_negotiation.cc:1035] Waiting for connection context
0809 15:38:20.066001 (+ 45us) server_negotiation.cc:299] Negotiation successful
0809 15:38:20.066083 (+ 82us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":699,"thread_start_us":116,"threads_started":1}
I20260809 15:38:20.066548 18710 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:38:20.060318 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37828 (local address 127.3.89.194:45639)
0809 15:38:20.060702 (+ 384us) server_negotiation.cc:206] Beginning negotiation
0809 15:38:20.060713 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0809 15:38:20.060736 (+ 23us) server_negotiation.cc:407] Connection header received
0809 15:38:20.060845 (+ 109us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 15:38:20.060850 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 15:38:20.061029 (+ 179us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 15:38:20.061179 (+ 150us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 15:38:20.061584 (+ 405us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:38:20.062745 (+ 1161us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:38:20.064578 (+ 1833us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:38:20.065406 (+ 828us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:38:20.065510 (+ 104us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:38:20.065647 (+ 137us) server_negotiation.cc:1035] Waiting for connection context
0809 15:38:20.066301 (+ 654us) server_negotiation.cc:299] Negotiation successful
0809 15:38:20.066416 (+ 115us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":290,"thread_start_us":93,"threads_started":1}
I20260809 15:38:20.068598 18624 tablet_service.cc:1467] Processing CreateTablet for tablet 2c3257975f264a548a7b27328a081a4e (DEFAULT_TABLE table=test-table [id=d23e7a7f86ab42efad4fee2ea8b0a852]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 15:38:20.068866 18482 tablet_service.cc:1467] Processing CreateTablet for tablet 2c3257975f264a548a7b27328a081a4e (DEFAULT_TABLE table=test-table [id=d23e7a7f86ab42efad4fee2ea8b0a852]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 15:38:20.069605 18624 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 2c3257975f264a548a7b27328a081a4e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 15:38:20.069763 18482 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 2c3257975f264a548a7b27328a081a4e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 15:38:20.075534 18712 tablet_bootstrap.cc:492] T 2c3257975f264a548a7b27328a081a4e P 273a5fb47ef740c196b1a55c87ed7466: Bootstrap starting.
I20260809 15:38:20.076936 18713 tablet_bootstrap.cc:492] T 2c3257975f264a548a7b27328a081a4e P 421cb337efc747e292ba53a026c44c02: Bootstrap starting.
I20260809 15:38:20.076943 18338 tablet_service.cc:1467] Processing CreateTablet for tablet 2c3257975f264a548a7b27328a081a4e (DEFAULT_TABLE table=test-table [id=d23e7a7f86ab42efad4fee2ea8b0a852]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 15:38:20.077669 18338 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 2c3257975f264a548a7b27328a081a4e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 15:38:20.078795 18712 tablet_bootstrap.cc:654] T 2c3257975f264a548a7b27328a081a4e P 273a5fb47ef740c196b1a55c87ed7466: Neither blocks nor log segments found. Creating new log.
I20260809 15:38:20.079360 18713 tablet_bootstrap.cc:654] T 2c3257975f264a548a7b27328a081a4e P 421cb337efc747e292ba53a026c44c02: Neither blocks nor log segments found. Creating new log.
I20260809 15:38:20.079716 18712 log.cc:824] T 2c3257975f264a548a7b27328a081a4e P 273a5fb47ef740c196b1a55c87ed7466: Log is configured to *not* fsync() on all Append() calls
I20260809 15:38:20.080120 18713 log.cc:824] T 2c3257975f264a548a7b27328a081a4e P 421cb337efc747e292ba53a026c44c02: Log is configured to *not* fsync() on all Append() calls
I20260809 15:38:20.081506 18712 tablet_bootstrap.cc:492] T 2c3257975f264a548a7b27328a081a4e P 273a5fb47ef740c196b1a55c87ed7466: No bootstrap required, opened a new log
I20260809 15:38:20.081736 18712 ts_tablet_manager.cc:1397] T 2c3257975f264a548a7b27328a081a4e P 273a5fb47ef740c196b1a55c87ed7466: Time spent bootstrapping tablet: real 0.006s user 0.004s sys 0.000s
I20260809 15:38:20.083546 18716 tablet_bootstrap.cc:492] T 2c3257975f264a548a7b27328a081a4e P b0a56d93c098468296a2000c9c5609ca: Bootstrap starting.
I20260809 15:38:20.085245 18713 tablet_bootstrap.cc:492] T 2c3257975f264a548a7b27328a081a4e P 421cb337efc747e292ba53a026c44c02: No bootstrap required, opened a new log
I20260809 15:38:20.085464 18713 ts_tablet_manager.cc:1397] T 2c3257975f264a548a7b27328a081a4e P 421cb337efc747e292ba53a026c44c02: Time spent bootstrapping tablet: real 0.009s user 0.004s sys 0.000s
I20260809 15:38:20.085584 18716 tablet_bootstrap.cc:654] T 2c3257975f264a548a7b27328a081a4e P b0a56d93c098468296a2000c9c5609ca: Neither blocks nor log segments found. Creating new log.
I20260809 15:38:20.085484 18712 raft_consensus.cc:348] T 2c3257975f264a548a7b27328a081a4e P 273a5fb47ef740c196b1a55c87ed7466 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "421cb337efc747e292ba53a026c44c02" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 40257 } } peers { permanent_uuid: "b0a56d93c098468296a2000c9c5609ca" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 44707 } } peers { permanent_uuid: "273a5fb47ef740c196b1a55c87ed7466" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 45639 } }
I20260809 15:38:20.085763 18712 raft_consensus.cc:374] T 2c3257975f264a548a7b27328a081a4e P 273a5fb47ef740c196b1a55c87ed7466 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 15:38:20.085843 18712 raft_consensus.cc:729] T 2c3257975f264a548a7b27328a081a4e P 273a5fb47ef740c196b1a55c87ed7466 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 273a5fb47ef740c196b1a55c87ed7466, State: Initialized, Role: FOLLOWER
I20260809 15:38:20.086293 18712 consensus_queue.cc:260] T 2c3257975f264a548a7b27328a081a4e P 273a5fb47ef740c196b1a55c87ed7466 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "421cb337efc747e292ba53a026c44c02" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 40257 } } peers { permanent_uuid: "b0a56d93c098468296a2000c9c5609ca" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 44707 } } peers { permanent_uuid: "273a5fb47ef740c196b1a55c87ed7466" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 45639 } }
I20260809 15:38:20.086454 18716 log.cc:824] T 2c3257975f264a548a7b27328a081a4e P b0a56d93c098468296a2000c9c5609ca: Log is configured to *not* fsync() on all Append() calls
I20260809 15:38:20.086997 18554 heartbeater.cc:503] Master 127.3.89.254:33497 was elected leader, sending a full tablet report...
I20260809 15:38:20.087373 18712 ts_tablet_manager.cc:1428] T 2c3257975f264a548a7b27328a081a4e P 273a5fb47ef740c196b1a55c87ed7466: Time spent starting tablet: real 0.006s user 0.006s sys 0.000s
I20260809 15:38:20.088405 18716 tablet_bootstrap.cc:492] T 2c3257975f264a548a7b27328a081a4e P b0a56d93c098468296a2000c9c5609ca: No bootstrap required, opened a new log
I20260809 15:38:20.088603 18716 ts_tablet_manager.cc:1397] T 2c3257975f264a548a7b27328a081a4e P b0a56d93c098468296a2000c9c5609ca: Time spent bootstrapping tablet: real 0.005s user 0.005s sys 0.000s
I20260809 15:38:20.089181 18713 raft_consensus.cc:348] T 2c3257975f264a548a7b27328a081a4e P 421cb337efc747e292ba53a026c44c02 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "421cb337efc747e292ba53a026c44c02" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 40257 } } peers { permanent_uuid: "b0a56d93c098468296a2000c9c5609ca" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 44707 } } peers { permanent_uuid: "273a5fb47ef740c196b1a55c87ed7466" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 45639 } }
I20260809 15:38:20.089413 18713 raft_consensus.cc:374] T 2c3257975f264a548a7b27328a081a4e P 421cb337efc747e292ba53a026c44c02 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 15:38:20.089483 18713 raft_consensus.cc:729] T 2c3257975f264a548a7b27328a081a4e P 421cb337efc747e292ba53a026c44c02 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 421cb337efc747e292ba53a026c44c02, State: Initialized, Role: FOLLOWER
I20260809 15:38:20.089902 18713 consensus_queue.cc:260] T 2c3257975f264a548a7b27328a081a4e P 421cb337efc747e292ba53a026c44c02 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "421cb337efc747e292ba53a026c44c02" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 40257 } } peers { permanent_uuid: "b0a56d93c098468296a2000c9c5609ca" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 44707 } } peers { permanent_uuid: "273a5fb47ef740c196b1a55c87ed7466" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 45639 } }
I20260809 15:38:20.090580 18694 heartbeater.cc:503] Master 127.3.89.254:33497 was elected leader, sending a full tablet report...
I20260809 15:38:20.090855 18713 ts_tablet_manager.cc:1428] T 2c3257975f264a548a7b27328a081a4e P 421cb337efc747e292ba53a026c44c02: Time spent starting tablet: real 0.005s user 0.001s sys 0.005s
I20260809 15:38:20.093058 18716 raft_consensus.cc:348] T 2c3257975f264a548a7b27328a081a4e P b0a56d93c098468296a2000c9c5609ca [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "421cb337efc747e292ba53a026c44c02" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 40257 } } peers { permanent_uuid: "b0a56d93c098468296a2000c9c5609ca" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 44707 } } peers { permanent_uuid: "273a5fb47ef740c196b1a55c87ed7466" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 45639 } }
I20260809 15:38:20.093340 18716 raft_consensus.cc:374] T 2c3257975f264a548a7b27328a081a4e P b0a56d93c098468296a2000c9c5609ca [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 15:38:20.093423 18716 raft_consensus.cc:729] T 2c3257975f264a548a7b27328a081a4e P b0a56d93c098468296a2000c9c5609ca [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b0a56d93c098468296a2000c9c5609ca, State: Initialized, Role: FOLLOWER
I20260809 15:38:20.093802 18716 consensus_queue.cc:260] T 2c3257975f264a548a7b27328a081a4e P b0a56d93c098468296a2000c9c5609ca [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "421cb337efc747e292ba53a026c44c02" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 40257 } } peers { permanent_uuid: "b0a56d93c098468296a2000c9c5609ca" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 44707 } } peers { permanent_uuid: "273a5fb47ef740c196b1a55c87ed7466" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 45639 } }
I20260809 15:38:20.094412 18410 heartbeater.cc:503] Master 127.3.89.254:33497 was elected leader, sending a full tablet report...
I20260809 15:38:20.094683 18716 ts_tablet_manager.cc:1428] T 2c3257975f264a548a7b27328a081a4e P b0a56d93c098468296a2000c9c5609ca: Time spent starting tablet: real 0.006s user 0.005s sys 0.000s
W20260809 15:38:20.257915 18695 tablet.cc:2369] T 2c3257975f264a548a7b27328a081a4e P 421cb337efc747e292ba53a026c44c02: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260809 15:38:20.293911 18555 tablet.cc:2369] T 2c3257975f264a548a7b27328a081a4e P 273a5fb47ef740c196b1a55c87ed7466: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260809 15:38:20.306684 18411 tablet.cc:2369] T 2c3257975f264a548a7b27328a081a4e P b0a56d93c098468296a2000c9c5609ca: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260809 15:38:20.334014 18717 raft_consensus.cc:482] T 2c3257975f264a548a7b27328a081a4e P 273a5fb47ef740c196b1a55c87ed7466 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 15:38:20.334290 18717 raft_consensus.cc:504] T 2c3257975f264a548a7b27328a081a4e P 273a5fb47ef740c196b1a55c87ed7466 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "421cb337efc747e292ba53a026c44c02" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 40257 } } peers { permanent_uuid: "b0a56d93c098468296a2000c9c5609ca" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 44707 } } peers { permanent_uuid: "273a5fb47ef740c196b1a55c87ed7466" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 45639 } }
I20260809 15:38:20.335639 18717 leader_election.cc:288] T 2c3257975f264a548a7b27328a081a4e P 273a5fb47ef740c196b1a55c87ed7466 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 421cb337efc747e292ba53a026c44c02 (127.3.89.195:40257), b0a56d93c098468296a2000c9c5609ca (127.3.89.193:44707)
I20260809 15:38:20.339777 18708 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:38:20.336069 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.89.194:56705 (local address 127.3.89.193:44707)
0809 15:38:20.336194 (+ 125us) server_negotiation.cc:206] Beginning negotiation
0809 15:38:20.336199 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 15:38:20.336412 (+ 213us) server_negotiation.cc:407] Connection header received
0809 15:38:20.336581 (+ 169us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 15:38:20.336584 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 15:38:20.336638 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 15:38:20.336724 (+ 86us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 15:38:20.337038 (+ 314us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:38:20.337598 (+ 560us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:38:20.338772 (+ 1174us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:38:20.339216 (+ 444us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:38:20.339351 (+ 135us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:38:20.339405 (+ 54us) server_negotiation.cc:1035] Waiting for connection context
0809 15:38:20.339590 (+ 185us) server_negotiation.cc:299] Negotiation successful
0809 15:38:20.339660 (+ 70us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":38}
I20260809 15:38:20.339793 18722 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:38:20.335944 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.89.193:44707 (local address 127.3.89.194:56705)
0809 15:38:20.336305 (+ 361us) negotiation.cc:107] Waiting for socket to connect
0809 15:38:20.336317 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0809 15:38:20.336454 (+ 137us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 15:38:20.336767 (+ 313us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 15:38:20.336770 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 15:38:20.336787 (+ 17us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 15:38:20.336922 (+ 135us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:38:20.336927 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:38:20.337712 (+ 785us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:38:20.337718 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:38:20.338641 (+ 923us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:38:20.338650 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:38:20.339341 (+ 691us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:38:20.339345 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:38:20.339440 (+ 95us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:38:20.339470 (+ 30us) client_negotiation.cc:769] Sending connection context
0809 15:38:20.339562 (+ 92us) client_negotiation.cc:240] Negotiation successful
0809 15:38:20.339635 (+ 73us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":310,"mutex_wait_us":71,"thread_start_us":81,"threads_started":1}
I20260809 15:38:20.339828 18706 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:38:20.335986 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.89.194:48641 (local address 127.3.89.195:40257)
0809 15:38:20.336164 (+ 178us) server_negotiation.cc:206] Beginning negotiation
0809 15:38:20.336169 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 15:38:20.336352 (+ 183us) server_negotiation.cc:407] Connection header received
0809 15:38:20.336521 (+ 169us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 15:38:20.336524 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 15:38:20.336577 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 15:38:20.336662 (+ 85us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 15:38:20.336945 (+ 283us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:38:20.337564 (+ 619us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:38:20.338774 (+ 1210us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:38:20.339270 (+ 496us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:38:20.339405 (+ 135us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:38:20.339464 (+ 59us) server_negotiation.cc:1035] Waiting for connection context
0809 15:38:20.339644 (+ 180us) server_negotiation.cc:299] Negotiation successful
0809 15:38:20.339716 (+ 72us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":85}
I20260809 15:38:20.339848 18721 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:38:20.335856 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.89.195:40257 (local address 127.3.89.194:48641)
0809 15:38:20.336253 (+ 397us) negotiation.cc:107] Waiting for socket to connect
0809 15:38:20.336265 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0809 15:38:20.336408 (+ 143us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 15:38:20.336685 (+ 277us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 15:38:20.336688 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 15:38:20.336704 (+ 16us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 15:38:20.336822 (+ 118us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:38:20.336836 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:38:20.337682 (+ 846us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:38:20.337687 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:38:20.338637 (+ 950us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:38:20.338647 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:38:20.339398 (+ 751us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:38:20.339402 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:38:20.339508 (+ 106us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:38:20.339528 (+ 20us) client_negotiation.cc:769] Sending connection context
0809 15:38:20.339628 (+ 100us) client_negotiation.cc:240] Negotiation successful
0809 15:38:20.339696 (+ 68us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":314,"mutex_wait_us":31,"thread_start_us":118,"threads_started":1}
I20260809 15:38:20.340804 18644 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2c3257975f264a548a7b27328a081a4e" candidate_uuid: "273a5fb47ef740c196b1a55c87ed7466" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "421cb337efc747e292ba53a026c44c02" is_pre_election: true
I20260809 15:38:20.340823 18359 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2c3257975f264a548a7b27328a081a4e" candidate_uuid: "273a5fb47ef740c196b1a55c87ed7466" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b0a56d93c098468296a2000c9c5609ca" is_pre_election: true
I20260809 15:38:20.341095 18359 raft_consensus.cc:2454] T 2c3257975f264a548a7b27328a081a4e P b0a56d93c098468296a2000c9c5609ca [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 273a5fb47ef740c196b1a55c87ed7466 in term 0.
I20260809 15:38:20.341109 18644 raft_consensus.cc:2454] T 2c3257975f264a548a7b27328a081a4e P 421cb337efc747e292ba53a026c44c02 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 273a5fb47ef740c196b1a55c87ed7466 in term 0.
I20260809 15:38:20.341564 18436 leader_election.cc:302] T 2c3257975f264a548a7b27328a081a4e P 273a5fb47ef740c196b1a55c87ed7466 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 273a5fb47ef740c196b1a55c87ed7466, b0a56d93c098468296a2000c9c5609ca; no voters:
I20260809 15:38:20.341843 18717 raft_consensus.cc:2781] T 2c3257975f264a548a7b27328a081a4e P 273a5fb47ef740c196b1a55c87ed7466 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260809 15:38:20.341925 18717 raft_consensus.cc:482] T 2c3257975f264a548a7b27328a081a4e P 273a5fb47ef740c196b1a55c87ed7466 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 15:38:20.341957 18717 raft_consensus.cc:3037] T 2c3257975f264a548a7b27328a081a4e P 273a5fb47ef740c196b1a55c87ed7466 [term 0 FOLLOWER]: Advancing to term 1
I20260809 15:38:20.343218 18717 raft_consensus.cc:504] T 2c3257975f264a548a7b27328a081a4e P 273a5fb47ef740c196b1a55c87ed7466 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "421cb337efc747e292ba53a026c44c02" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 40257 } } peers { permanent_uuid: "b0a56d93c098468296a2000c9c5609ca" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 44707 } } peers { permanent_uuid: "273a5fb47ef740c196b1a55c87ed7466" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 45639 } }
I20260809 15:38:20.343602 18717 leader_election.cc:288] T 2c3257975f264a548a7b27328a081a4e P 273a5fb47ef740c196b1a55c87ed7466 [CANDIDATE]: Term 1 election: Requested vote from peers 421cb337efc747e292ba53a026c44c02 (127.3.89.195:40257), b0a56d93c098468296a2000c9c5609ca (127.3.89.193:44707)
I20260809 15:38:20.343971 18644 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2c3257975f264a548a7b27328a081a4e" candidate_uuid: "273a5fb47ef740c196b1a55c87ed7466" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "421cb337efc747e292ba53a026c44c02"
I20260809 15:38:20.344050 18359 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2c3257975f264a548a7b27328a081a4e" candidate_uuid: "273a5fb47ef740c196b1a55c87ed7466" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b0a56d93c098468296a2000c9c5609ca"
I20260809 15:38:20.344105 18644 raft_consensus.cc:3037] T 2c3257975f264a548a7b27328a081a4e P 421cb337efc747e292ba53a026c44c02 [term 0 FOLLOWER]: Advancing to term 1
I20260809 15:38:20.344187 18359 raft_consensus.cc:3037] T 2c3257975f264a548a7b27328a081a4e P b0a56d93c098468296a2000c9c5609ca [term 0 FOLLOWER]: Advancing to term 1
I20260809 15:38:20.345216 18359 raft_consensus.cc:2454] T 2c3257975f264a548a7b27328a081a4e P b0a56d93c098468296a2000c9c5609ca [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 273a5fb47ef740c196b1a55c87ed7466 in term 1.
I20260809 15:38:20.345250 18644 raft_consensus.cc:2454] T 2c3257975f264a548a7b27328a081a4e P 421cb337efc747e292ba53a026c44c02 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 273a5fb47ef740c196b1a55c87ed7466 in term 1.
I20260809 15:38:20.345496 18436 leader_election.cc:302] T 2c3257975f264a548a7b27328a081a4e P 273a5fb47ef740c196b1a55c87ed7466 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 273a5fb47ef740c196b1a55c87ed7466, b0a56d93c098468296a2000c9c5609ca; no voters:
I20260809 15:38:20.345654 18717 raft_consensus.cc:2781] T 2c3257975f264a548a7b27328a081a4e P 273a5fb47ef740c196b1a55c87ed7466 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 15:38:20.345846 18717 raft_consensus.cc:686] T 2c3257975f264a548a7b27328a081a4e P 273a5fb47ef740c196b1a55c87ed7466 [term 1 LEADER]: Becoming Leader. State: Replica: 273a5fb47ef740c196b1a55c87ed7466, State: Running, Role: LEADER
I20260809 15:38:20.346160 18717 consensus_queue.cc:237] T 2c3257975f264a548a7b27328a081a4e P 273a5fb47ef740c196b1a55c87ed7466 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "421cb337efc747e292ba53a026c44c02" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 40257 } } peers { permanent_uuid: "b0a56d93c098468296a2000c9c5609ca" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 44707 } } peers { permanent_uuid: "273a5fb47ef740c196b1a55c87ed7466" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 45639 } }
I20260809 15:38:20.349196 18203 catalog_manager.cc:5495] T 2c3257975f264a548a7b27328a081a4e P 273a5fb47ef740c196b1a55c87ed7466 reported cstate change: term changed from 0 to 1, leader changed from <none> to 273a5fb47ef740c196b1a55c87ed7466 (127.3.89.194). New cstate: current_term: 1 leader_uuid: "273a5fb47ef740c196b1a55c87ed7466" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "421cb337efc747e292ba53a026c44c02" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 40257 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b0a56d93c098468296a2000c9c5609ca" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 44707 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "273a5fb47ef740c196b1a55c87ed7466" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 45639 } health_report { overall_health: HEALTHY } } }
I20260809 15:38:20.430670 18710 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:38:20.426708 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37832 (local address 127.3.89.194:45639)
0809 15:38:20.426860 (+ 152us) server_negotiation.cc:206] Beginning negotiation
0809 15:38:20.426865 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 15:38:20.426881 (+ 16us) server_negotiation.cc:407] Connection header received
0809 15:38:20.427030 (+ 149us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 15:38:20.427033 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 15:38:20.427089 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 15:38:20.427208 (+ 119us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 15:38:20.427469 (+ 261us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:38:20.427976 (+ 507us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:38:20.429092 (+ 1116us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:38:20.429341 (+ 249us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:38:20.429483 (+ 142us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:38:20.429843 (+ 360us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 15:38:20.430253 (+ 410us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 15:38:20.430394 (+ 141us) server_negotiation.cc:1035] Waiting for connection context
0809 15:38:20.430455 (+ 61us) server_negotiation.cc:299] Negotiation successful
0809 15:38:20.430521 (+ 66us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":69}
I20260809 15:38:20.436774 18359 raft_consensus.cc:1261] T 2c3257975f264a548a7b27328a081a4e P b0a56d93c098468296a2000c9c5609ca [term 1 FOLLOWER]: Refusing update from remote peer 273a5fb47ef740c196b1a55c87ed7466: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 15:38:20.436800 18644 raft_consensus.cc:1261] T 2c3257975f264a548a7b27328a081a4e P 421cb337efc747e292ba53a026c44c02 [term 1 FOLLOWER]: Refusing update from remote peer 273a5fb47ef740c196b1a55c87ed7466: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 15:38:20.437489 18717 consensus_queue.cc:1035] T 2c3257975f264a548a7b27328a081a4e P 273a5fb47ef740c196b1a55c87ed7466 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b0a56d93c098468296a2000c9c5609ca" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 44707 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 15:38:20.437780 18723 consensus_queue.cc:1035] T 2c3257975f264a548a7b27328a081a4e P 273a5fb47ef740c196b1a55c87ed7466 [LEADER]: Connected to new peer: Peer: permanent_uuid: "421cb337efc747e292ba53a026c44c02" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 40257 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 15:38:20.448326 18727 mvcc.cc:205] Tried to move back new op lower bound from 7316643432181637120 to 7316643431820374016. Current Snapshot: MvccSnapshot[applied={T|T < 7316643432181637120 or (T in {7316643432181637120})}]
I20260809 15:38:20.455574 18204 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:47444:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 15:38:20.455713 18204 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:47444:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 15:38:20.458102 18204 catalog_manager.cc:5782] T 00000000000000000000000000000000 P f5ecfca57914460d8be340f7b52004b5: Sending DeleteTablet for 3 replicas of tablet 2c3257975f264a548a7b27328a081a4e
I20260809 15:38:20.458844 18624 tablet_service.cc:1514] Processing DeleteTablet for tablet 2c3257975f264a548a7b27328a081a4e with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 15:38:20 UTC) from {username='kudu', principal='kudu/127.3.89.254@KRBTEST.COM'} at 127.0.0.1:33674
I20260809 15:38:20.458847 18338 tablet_service.cc:1514] Processing DeleteTablet for tablet 2c3257975f264a548a7b27328a081a4e with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 15:38:20 UTC) from {username='kudu', principal='kudu/127.3.89.254@KRBTEST.COM'} at 127.0.0.1:56864
I20260809 15:38:20.459339 18482 tablet_service.cc:1514] Processing DeleteTablet for tablet 2c3257975f264a548a7b27328a081a4e with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 15:38:20 UTC) from {username='kudu', principal='kudu/127.3.89.254@KRBTEST.COM'} at 127.0.0.1:37828
I20260809 15:38:20.459386 3431 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskm314YR/build/debug/bin/kudu with pid 18271
I20260809 15:38:20.461521 18738 tablet_replica.cc:330] T 2c3257975f264a548a7b27328a081a4e P 273a5fb47ef740c196b1a55c87ed7466: stopping tablet replica
I20260809 15:38:20.461825 18738 raft_consensus.cc:2229] T 2c3257975f264a548a7b27328a081a4e P 273a5fb47ef740c196b1a55c87ed7466 [term 1 LEADER]: Raft consensus shutting down.
I20260809 15:38:20.462177 18738 raft_consensus.cc:2258] T 2c3257975f264a548a7b27328a081a4e P 273a5fb47ef740c196b1a55c87ed7466 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 15:38:20.463598 18738 ts_tablet_manager.cc:1905] T 2c3257975f264a548a7b27328a081a4e P 273a5fb47ef740c196b1a55c87ed7466: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 15:38:20.464609 18737 tablet_replica.cc:330] T 2c3257975f264a548a7b27328a081a4e P 421cb337efc747e292ba53a026c44c02: stopping tablet replica
I20260809 15:38:20.464908 18737 raft_consensus.cc:2229] T 2c3257975f264a548a7b27328a081a4e P 421cb337efc747e292ba53a026c44c02 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 15:38:20.465219 18737 raft_consensus.cc:2258] T 2c3257975f264a548a7b27328a081a4e P 421cb337efc747e292ba53a026c44c02 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 15:38:20.466571 18737 ts_tablet_manager.cc:1905] T 2c3257975f264a548a7b27328a081a4e P 421cb337efc747e292ba53a026c44c02: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 15:38:20.467216 18738 ts_tablet_manager.cc:1918] T 2c3257975f264a548a7b27328a081a4e P 273a5fb47ef740c196b1a55c87ed7466: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 15:38:20.467309 18738 log.cc:1196] T 2c3257975f264a548a7b27328a081a4e P 273a5fb47ef740c196b1a55c87ed7466: Deleting WAL directory at /tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786289732823074-3431-0/minicluster-data/ts-1/wal/wals/2c3257975f264a548a7b27328a081a4e
I20260809 15:38:20.467666 18738 ts_tablet_manager.cc:1939] T 2c3257975f264a548a7b27328a081a4e P 273a5fb47ef740c196b1a55c87ed7466: Deleting consensus metadata
I20260809 15:38:20.468938 18191 catalog_manager.cc:4841] TS 273a5fb47ef740c196b1a55c87ed7466 (127.3.89.194:45639): tablet 2c3257975f264a548a7b27328a081a4e (table test-table [id=d23e7a7f86ab42efad4fee2ea8b0a852]) successfully deleted
I20260809 15:38:20.469169 18737 ts_tablet_manager.cc:1918] T 2c3257975f264a548a7b27328a081a4e P 421cb337efc747e292ba53a026c44c02: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 15:38:20.469271 18737 log.cc:1196] T 2c3257975f264a548a7b27328a081a4e P 421cb337efc747e292ba53a026c44c02: Deleting WAL directory at /tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786289732823074-3431-0/minicluster-data/ts-2/wal/wals/2c3257975f264a548a7b27328a081a4e
I20260809 15:38:20.469604 18737 ts_tablet_manager.cc:1939] T 2c3257975f264a548a7b27328a081a4e P 421cb337efc747e292ba53a026c44c02: Deleting consensus metadata
I20260809 15:38:20.470780 18189 catalog_manager.cc:4841] TS 421cb337efc747e292ba53a026c44c02 (127.3.89.195:40257): tablet 2c3257975f264a548a7b27328a081a4e (table test-table [id=d23e7a7f86ab42efad4fee2ea8b0a852]) successfully deleted
I20260809 15:38:20.471088 3431 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskm314YR/build/debug/bin/kudu with pid 18416
W20260809 15:38:20.471335 18192 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.3.89.193:44707 (error 108)
I20260809 15:38:20.472918 18709 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:38:20.472643 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.89.193:44707 (local address 127.0.0.1:56876)
0809 15:38:20.472774 (+ 131us) negotiation.cc:107] Waiting for socket to connect
0809 15:38:20.472848 (+ 74us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.89.193:44707: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":52}
W20260809 15:38:20.473119 18192 catalog_manager.cc:4584] TS b0a56d93c098468296a2000c9c5609ca (127.3.89.193:44707): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 2c3257975f264a548a7b27328a081a4e: Network error: Client connection negotiation failed: client connection to 127.3.89.193:44707: connect: Connection refused (error 111)
I20260809 15:38:20.479228 3431 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskm314YR/build/debug/bin/kudu with pid 18559
I20260809 15:38:20.486608 3431 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskm314YR/build/debug/bin/kudu with pid 18172
I20260809 15:38:20.493875 3431 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskm314YR/build/debug/bin/kudu
/tmp/dist-test-taskm314YR/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786289732823074-3431-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786289732823074-3431-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786289732823074-3431-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786289732823074-3431-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.3.89.254:33497
--webserver_interface=127.3.89.254
--webserver_port=38433
--keytab_file=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786289732823074-3431-0/krb5kdc/kudu_127.3.89.254.keytab
--principal=kudu/127.3.89.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.3.89.212:43771
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.3.89.254:33497
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--ipki_private_key_password_cmd=echo foo with env {KRB5CCNAME=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786289732823074-3431-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786289732823074-3431-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786289732823074-3431-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 15:38:20.606263 18740 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 15:38:20.606519 18740 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 15:38:20.606557 18740 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 15:38:20.610546 18740 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 15:38:20.610601 18740 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 15:38:20.610620 18740 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 15:38:20.610637 18740 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 15:38:20.610683 18740 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 15:38:20.615741 18740 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.3.89.212:43771
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786289732823074-3431-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786289732823074-3431-0/minicluster-data/master-0/wal
--ipki_private_key_password_cmd=echo foo
--ipki_ca_key_size=768
--master_addresses=127.3.89.254:33497
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786289732823074-3431-0/krb5kdc/kudu_127.3.89.254.keytab
--principal=kudu/127.3.89.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.3.89.254:33497
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786289732823074-3431-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.3.89.254
--webserver_port=38433
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.18740
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786289732823074-3431-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 09 Aug 2026 14:46:55 UTC on ccycloud.quasar-yswkmf.root.comops.site
build id 4576540
I20260809 15:38:20.616704 18740 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 15:38:20.617784 18740 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 15:38:20.631515 18747 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 15:38:20.631515 18748 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 15:38:20.631589 18750 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 15:38:20.631675 18740 server_base.cc:1034] running on GCE node
I20260809 15:38:20.632454 18740 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 15:38:20.633617 18740 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 15:38:20.634859 18740 hybrid_clock.cc:648] HybridClock initialized: now 1786289900634848 us; error 32 us; skew 500 ppm
Aug 09 15:38:20 dist-test-slave-2w3w krb5kdc[18158](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786289900, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.89.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 15:38:20.637830 18740 init.cc:377] Logged in from keytab as kudu/127.3.89.254@KRBTEST.COM (short username kudu)
I20260809 15:38:20.638852 18740 webserver.cc:460] Webserver started at http://127.3.89.254:38433/ using document root <none> and password file <none>
I20260809 15:38:20.639474 18740 fs_manager.cc:359] Metadata directory not provided
I20260809 15:38:20.639599 18740 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 15:38:20.643465 18740 fs_manager.cc:709] Time spent opening directory manager: real 0.002s user 0.004s sys 0.000s
I20260809 15:38:20.645581 18758 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 15:38:20.646801 18740 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260809 15:38:20.646948 18740 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786289732823074-3431-0/minicluster-data/master-0/data,/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786289732823074-3431-0/minicluster-data/master-0/wal
uuid: "f5ecfca57914460d8be340f7b52004b5"
format_stamp: "Formatted at 2026-08-09 15:38:19 on dist-test-slave-2w3w"
server_key: "aaf2376a0c2aedc840940ad38cd4d160"
server_key_iv: "e61ba99cabf8f23507a5d3c83dc759f0"
server_key_version: "encryptionkey@0"
I20260809 15:38:20.647617 18740 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786289732823074-3431-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786289732823074-3431-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786289732823074-3431-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 15:38:20.681623 18740 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 15:38:20.685158 18740 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 15:38:20.685387 18740 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 15:38:20.692893 18740 rpc_server.cc:307] RPC server started. Bound to: 127.3.89.254:33497
I20260809 15:38:20.693199 18812 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.3.89.254:33497 every 8 connection(s)
I20260809 15:38:20.694083 18740 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786289732823074-3431-0/minicluster-data/master-0/data/info.pb
I20260809 15:38:20.698235 18813 sys_catalog.cc:263] Verifying existing consensus state
I20260809 15:38:20.699818 18813 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f5ecfca57914460d8be340f7b52004b5: Bootstrap starting.
I20260809 15:38:20.700755 3431 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskm314YR/build/debug/bin/kudu as pid 18740
I20260809 15:38:20.701180 3431 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskm314YR/build/debug/bin/kudu
/tmp/dist-test-taskm314YR/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786289732823074-3431-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786289732823074-3431-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786289732823074-3431-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786289732823074-3431-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.3.89.193:44707
--local_ip_for_outbound_sockets=127.3.89.193
--tserver_master_addrs=127.3.89.254:33497
--webserver_port=42811
--webserver_interface=127.3.89.193
--keytab_file=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786289732823074-3431-0/krb5kdc/kudu_127.3.89.193.keytab
--principal=kudu/127.3.89.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.3.89.212:43771
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786289732823074-3431-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786289732823074-3431-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786289732823074-3431-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260809 15:38:20.722002 18813 log.cc:824] T 00000000000000000000000000000000 P f5ecfca57914460d8be340f7b52004b5: Log is configured to *not* fsync() on all Append() calls
I20260809 15:38:20.732533 18813 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f5ecfca57914460d8be340f7b52004b5: Bootstrap replayed 1/1 log segments. Stats: ops{read=8 overwritten=0 applied=8 ignored=0} inserts{seen=5 ignored=0} mutations{seen=4 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260809 15:38:20.732937 18813 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f5ecfca57914460d8be340f7b52004b5: Bootstrap complete.
I20260809 15:38:20.738508 18813 raft_consensus.cc:348] T 00000000000000000000000000000000 P f5ecfca57914460d8be340f7b52004b5 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f5ecfca57914460d8be340f7b52004b5" member_type: VOTER last_known_addr { host: "127.3.89.254" port: 33497 } }
I20260809 15:38:20.739049 18813 raft_consensus.cc:729] T 00000000000000000000000000000000 P f5ecfca57914460d8be340f7b52004b5 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: f5ecfca57914460d8be340f7b52004b5, State: Initialized, Role: FOLLOWER
I20260809 15:38:20.739557 18813 consensus_queue.cc:260] T 00000000000000000000000000000000 P f5ecfca57914460d8be340f7b52004b5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 8, Last appended: 1.8, Last appended by leader: 8, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f5ecfca57914460d8be340f7b52004b5" member_type: VOTER last_known_addr { host: "127.3.89.254" port: 33497 } }
I20260809 15:38:20.739696 18813 raft_consensus.cc:388] T 00000000000000000000000000000000 P f5ecfca57914460d8be340f7b52004b5 [term 1 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 15:38:20.739776 18813 raft_consensus.cc:482] T 00000000000000000000000000000000 P f5ecfca57914460d8be340f7b52004b5 [term 1 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 15:38:20.739867 18813 raft_consensus.cc:3037] T 00000000000000000000000000000000 P f5ecfca57914460d8be340f7b52004b5 [term 1 FOLLOWER]: Advancing to term 2
I20260809 15:38:20.742235 18813 raft_consensus.cc:504] T 00000000000000000000000000000000 P f5ecfca57914460d8be340f7b52004b5 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f5ecfca57914460d8be340f7b52004b5" member_type: VOTER last_known_addr { host: "127.3.89.254" port: 33497 } }
I20260809 15:38:20.742586 18813 leader_election.cc:302] T 00000000000000000000000000000000 P f5ecfca57914460d8be340f7b52004b5 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: f5ecfca57914460d8be340f7b52004b5; no voters:
I20260809 15:38:20.742941 18813 leader_election.cc:288] T 00000000000000000000000000000000 P f5ecfca57914460d8be340f7b52004b5 [CANDIDATE]: Term 2 election: Requested vote from peers
I20260809 15:38:20.743094 18817 raft_consensus.cc:2781] T 00000000000000000000000000000000 P f5ecfca57914460d8be340f7b52004b5 [term 2 FOLLOWER]: Leader election won for term 2
I20260809 15:38:20.743358 18817 raft_consensus.cc:686] T 00000000000000000000000000000000 P f5ecfca57914460d8be340f7b52004b5 [term 2 LEADER]: Becoming Leader. State: Replica: f5ecfca57914460d8be340f7b52004b5, State: Running, Role: LEADER
I20260809 15:38:20.743654 18817 consensus_queue.cc:237] T 00000000000000000000000000000000 P f5ecfca57914460d8be340f7b52004b5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 1.8, Last appended by leader: 8, Current term: 2, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f5ecfca57914460d8be340f7b52004b5" member_type: VOTER last_known_addr { host: "127.3.89.254" port: 33497 } }
I20260809 15:38:20.744270 18813 sys_catalog.cc:564] T 00000000000000000000000000000000 P f5ecfca57914460d8be340f7b52004b5 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 15:38:20.744606 18819 sys_catalog.cc:455] T 00000000000000000000000000000000 P f5ecfca57914460d8be340f7b52004b5 [sys.catalog]: SysCatalogTable state changed. Reason: New leader f5ecfca57914460d8be340f7b52004b5. Latest consensus state: current_term: 2 leader_uuid: "f5ecfca57914460d8be340f7b52004b5" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f5ecfca57914460d8be340f7b52004b5" member_type: VOTER last_known_addr { host: "127.3.89.254" port: 33497 } } }
I20260809 15:38:20.744745 18819 sys_catalog.cc:458] T 00000000000000000000000000000000 P f5ecfca57914460d8be340f7b52004b5 [sys.catalog]: This master's current role is: LEADER
I20260809 15:38:20.745012 18818 sys_catalog.cc:455] T 00000000000000000000000000000000 P f5ecfca57914460d8be340f7b52004b5 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 2 leader_uuid: "f5ecfca57914460d8be340f7b52004b5" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f5ecfca57914460d8be340f7b52004b5" member_type: VOTER last_known_addr { host: "127.3.89.254" port: 33497 } } }
I20260809 15:38:20.745108 18818 sys_catalog.cc:458] T 00000000000000000000000000000000 P f5ecfca57914460d8be340f7b52004b5 [sys.catalog]: This master's current role is: LEADER
I20260809 15:38:20.745374 18822 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 15:38:20.749943 18822 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 15:38:20.750844 18822 catalog_manager.cc:1259] Loaded cluster ID: 6b5d21f311dd4f3e809e5ae566290b34
I20260809 15:38:20.750908 18822 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 15:38:20.752359 18822 catalog_manager.cc:1314] IPKI private key was stored in cleartext, attempting to encrypt it.
I20260809 15:38:20.757282 18822 catalog_manager.cc:1324] IPKI private key successfully encrypted.
I20260809 15:38:20.762584 18822 catalog_manager.cc:1504] Loading token signing keys...
I20260809 15:38:20.763325 18822 catalog_manager.cc:5879] T 00000000000000000000000000000000 P f5ecfca57914460d8be340f7b52004b5: Loaded TSK: 0
I20260809 15:38:20.764489 18822 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260809 15:38:20.826618 18815 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 15:38:20.826874 18815 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 15:38:20.826947 18815 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 15:38:20.831689 18815 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 15:38:20.831782 18815 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 15:38:20.831874 18815 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.3.89.193
I20260809 15:38:20.837317 18815 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.3.89.212:43771
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786289732823074-3431-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786289732823074-3431-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786289732823074-3431-0/krb5kdc/kudu_127.3.89.193.keytab
--principal=kudu/127.3.89.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.3.89.193:44707
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786289732823074-3431-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.3.89.193
--webserver_port=42811
--tserver_master_addrs=127.3.89.254:33497
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.18815
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.3.89.193
--log_dir=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786289732823074-3431-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 09 Aug 2026 14:46:55 UTC on ccycloud.quasar-yswkmf.root.comops.site
build id 4576540
I20260809 15:38:20.838397 18815 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 15:38:20.839471 18815 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 15:38:20.849179 18843 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 15:38:20.849148 18844 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 15:38:20.849156 18846 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 15:38:20.849697 18815 server_base.cc:1034] running on GCE node
I20260809 15:38:20.850132 18815 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 15:38:20.850876 18815 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 15:38:20.852056 18815 hybrid_clock.cc:648] HybridClock initialized: now 1786289900852035 us; error 38 us; skew 500 ppm
Aug 09 15:38:20 dist-test-slave-2w3w krb5kdc[18158](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786289900, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.89.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 15:38:20.854808 18815 init.cc:377] Logged in from keytab as kudu/127.3.89.193@KRBTEST.COM (short username kudu)
I20260809 15:38:20.855778 18815 webserver.cc:460] Webserver started at http://127.3.89.193:42811/ using document root <none> and password file <none>
I20260809 15:38:20.856335 18815 fs_manager.cc:359] Metadata directory not provided
I20260809 15:38:20.856410 18815 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 15:38:20.860361 18815 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.001s sys 0.002s
I20260809 15:38:20.862416 18853 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 15:38:20.863658 18815 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260809 15:38:20.863745 18815 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786289732823074-3431-0/minicluster-data/ts-0/data,/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786289732823074-3431-0/minicluster-data/ts-0/wal
uuid: "b0a56d93c098468296a2000c9c5609ca"
format_stamp: "Formatted at 2026-08-09 15:38:19 on dist-test-slave-2w3w"
server_key: "f0ee155f811f0aa13c8b3b8f3e49f2bb"
server_key_iv: "5e2a5a61d21e9b6c20da06f34bb7cbf2"
server_key_version: "encryptionkey@0"
I20260809 15:38:20.864277 18815 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786289732823074-3431-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786289732823074-3431-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786289732823074-3431-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 15:38:20.883996 18815 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 15:38:20.887101 18815 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 15:38:20.887295 18815 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 15:38:20.887930 18815 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 15:38:20.889005 18860 ts_tablet_manager.cc:536] Loading tablet metadata (0/1 complete)
I20260809 15:38:20.892961 18815 ts_tablet_manager.cc:579] Loaded tablet metadata (1 total tablets, 1 live tablets)
I20260809 15:38:20.893023 18815 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.004s user 0.001s sys 0.000s
I20260809 15:38:20.893092 18815 ts_tablet_manager.cc:594] Registering tablets (0/1 complete)
I20260809 15:38:20.894706 18815 ts_tablet_manager.cc:610] Registered 1 tablets
I20260809 15:38:20.894783 18815 ts_tablet_manager.cc:589] Time spent register tablets: real 0.002s user 0.000s sys 0.000s
I20260809 15:38:20.895059 18860 tablet_bootstrap.cc:492] T 2c3257975f264a548a7b27328a081a4e P b0a56d93c098468296a2000c9c5609ca: Bootstrap starting.
I20260809 15:38:20.905562 18815 rpc_server.cc:307] RPC server started. Bound to: 127.3.89.193:44707
I20260809 15:38:20.905675 18978 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.3.89.193:44707 every 8 connection(s)
I20260809 15:38:20.906739 18815 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786289732823074-3431-0/minicluster-data/ts-0/data/info.pb
I20260809 15:38:20.909704 3431 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskm314YR/build/debug/bin/kudu as pid 18815
I20260809 15:38:20.910172 3431 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskm314YR/build/debug/bin/kudu
/tmp/dist-test-taskm314YR/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786289732823074-3431-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786289732823074-3431-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786289732823074-3431-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786289732823074-3431-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.3.89.194:45639
--local_ip_for_outbound_sockets=127.3.89.194
--tserver_master_addrs=127.3.89.254:33497
--webserver_port=40495
--webserver_interface=127.3.89.194
--keytab_file=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786289732823074-3431-0/krb5kdc/kudu_127.3.89.194.keytab
--principal=kudu/127.3.89.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.3.89.212:43771
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786289732823074-3431-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786289732823074-3431-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786289732823074-3431-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260809 15:38:20.915503 18860 log.cc:824] T 2c3257975f264a548a7b27328a081a4e P b0a56d93c098468296a2000c9c5609ca: Log is configured to *not* fsync() on all Append() calls
Aug 09 15:38:20 dist-test-slave-2w3w krb5kdc[18158](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786289900, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.89.193@KRBTEST.COM for kudu/127.3.89.254@KRBTEST.COM
I20260809 15:38:20.923182 18860 tablet_bootstrap.cc:492] T 2c3257975f264a548a7b27328a081a4e P b0a56d93c098468296a2000c9c5609ca: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=1 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260809 15:38:20.923571 18860 tablet_bootstrap.cc:492] T 2c3257975f264a548a7b27328a081a4e P b0a56d93c098468296a2000c9c5609ca: Bootstrap complete.
I20260809 15:38:20.924901 18860 ts_tablet_manager.cc:1397] T 2c3257975f264a548a7b27328a081a4e P b0a56d93c098468296a2000c9c5609ca: Time spent bootstrapping tablet: real 0.030s user 0.021s sys 0.007s
I20260809 15:38:20.926394 18982 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:38:20.908889 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.89.254:33497 (local address 127.3.89.193:60439)
0809 15:38:20.909483 (+ 594us) negotiation.cc:107] Waiting for socket to connect
0809 15:38:20.909544 (+ 61us) client_negotiation.cc:174] Beginning negotiation
0809 15:38:20.910636 (+ 1092us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 15:38:20.912130 (+ 1494us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 15:38:20.912144 (+ 14us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 15:38:20.912598 (+ 454us) client_negotiation.cc:189] Negotiated authn=SASL
0809 15:38:20.913333 (+ 735us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:38:20.913361 (+ 28us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:38:20.914678 (+ 1317us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:38:20.914682 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:38:20.915994 (+ 1312us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:38:20.916003 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:38:20.917013 (+ 1010us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:38:20.917020 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:38:20.917265 (+ 245us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:38:20.918024 (+ 759us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 15:38:20.918050 (+ 26us) client_negotiation.cc:656] calling sasl_client_start()
0809 15:38:20.920056 (+ 2006us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 15:38:20.922923 (+ 2867us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 15:38:20.922930 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 15:38:20.922941 (+ 11us) client_negotiation.cc:760] calling sasl_client_step()
0809 15:38:20.923300 (+ 359us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 15:38:20.923772 (+ 472us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 15:38:20.923777 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 15:38:20.923780 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0809 15:38:20.923975 (+ 195us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 15:38:20.924731 (+ 756us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 15:38:20.924737 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 15:38:20.925009 (+ 272us) client_negotiation.cc:769] Sending connection context
0809 15:38:20.925194 (+ 185us) client_negotiation.cc:240] Negotiation successful
0809 15:38:20.925508 (+ 314us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":359,"thread_start_us":193,"threads_started":1}
I20260809 15:38:20.926442 18981 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:38:20.908870 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.89.193:60439 (local address 127.3.89.254:33497)
0809 15:38:20.909225 (+ 355us) server_negotiation.cc:206] Beginning negotiation
0809 15:38:20.909246 (+ 21us) server_negotiation.cc:399] Waiting for connection header
0809 15:38:20.910983 (+ 1737us) server_negotiation.cc:407] Connection header received
0809 15:38:20.911268 (+ 285us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 15:38:20.911282 (+ 14us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 15:38:20.911649 (+ 367us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 15:38:20.911976 (+ 327us) server_negotiation.cc:226] Negotiated authn=SASL
0809 15:38:20.913564 (+ 1588us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:38:20.914508 (+ 944us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:38:20.916131 (+ 1623us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:38:20.916845 (+ 714us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:38:20.917017 (+ 172us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:38:20.920275 (+ 3258us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 15:38:20.920284 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 15:38:20.920305 (+ 21us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 15:38:20.920343 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0809 15:38:20.922780 (+ 2437us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 15:38:20.923447 (+ 667us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 15:38:20.923455 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 15:38:20.923464 (+ 9us) server_negotiation.cc:967] calling sasl_server_step()
0809 15:38:20.923634 (+ 170us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 15:38:20.924117 (+ 483us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 15:38:20.924123 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 15:38:20.924128 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0809 15:38:20.924594 (+ 466us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 15:38:20.924728 (+ 134us) server_negotiation.cc:1035] Waiting for connection context
0809 15:38:20.925371 (+ 643us) server_negotiation.cc:299] Negotiation successful
0809 15:38:20.925660 (+ 289us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":217,"thread_start_us":120,"threads_started":1}
I20260809 15:38:20.927732 18860 raft_consensus.cc:348] T 2c3257975f264a548a7b27328a081a4e P b0a56d93c098468296a2000c9c5609ca [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "421cb337efc747e292ba53a026c44c02" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 40257 } } peers { permanent_uuid: "b0a56d93c098468296a2000c9c5609ca" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 44707 } } peers { permanent_uuid: "273a5fb47ef740c196b1a55c87ed7466" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 45639 } }
I20260809 15:38:20.928200 18860 raft_consensus.cc:729] T 2c3257975f264a548a7b27328a081a4e P b0a56d93c098468296a2000c9c5609ca [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: b0a56d93c098468296a2000c9c5609ca, State: Initialized, Role: FOLLOWER
I20260809 15:38:20.928664 18860 consensus_queue.cc:260] T 2c3257975f264a548a7b27328a081a4e P b0a56d93c098468296a2000c9c5609ca [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "421cb337efc747e292ba53a026c44c02" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 40257 } } peers { permanent_uuid: "b0a56d93c098468296a2000c9c5609ca" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 44707 } } peers { permanent_uuid: "273a5fb47ef740c196b1a55c87ed7466" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 45639 } }
I20260809 15:38:20.929373 18979 heartbeater.cc:347] Connected to a master server at 127.3.89.254:33497
I20260809 15:38:20.929598 18979 heartbeater.cc:464] Registering TS with master...
I20260809 15:38:20.929711 18860 ts_tablet_manager.cc:1428] T 2c3257975f264a548a7b27328a081a4e P b0a56d93c098468296a2000c9c5609ca: Time spent starting tablet: real 0.005s user 0.006s sys 0.000s
I20260809 15:38:20.930071 18979 heartbeater.cc:511] Master 127.3.89.254:33497 requested a full tablet report, sending...
I20260809 15:38:20.932423 18776 ts_manager.cc:194] Registered new tserver with Master: b0a56d93c098468296a2000c9c5609ca (127.3.89.193:44707)
I20260809 15:38:20.935662 18776 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.3.89.193@KRBTEST.COM'} at 127.3.89.193:60439
I20260809 15:38:20.937923 18979 heartbeater.cc:503] Master 127.3.89.254:33497 was elected leader, sending a full tablet report...
Aug 09 15:38:20 dist-test-slave-2w3w krb5kdc[18158](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786289900, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.89.254@KRBTEST.COM for kudu/127.3.89.193@KRBTEST.COM
I20260809 15:38:20.946919 18987 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:38:20.935017 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.89.193:44707 (local address 127.0.0.1:56888)
0809 15:38:20.935764 (+ 747us) negotiation.cc:107] Waiting for socket to connect
0809 15:38:20.935814 (+ 50us) client_negotiation.cc:174] Beginning negotiation
0809 15:38:20.935955 (+ 141us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 15:38:20.936660 (+ 705us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 15:38:20.936666 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 15:38:20.937838 (+ 1172us) client_negotiation.cc:189] Negotiated authn=SASL
0809 15:38:20.937964 (+ 126us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:38:20.937972 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:38:20.939226 (+ 1254us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:38:20.939231 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:38:20.939998 (+ 767us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:38:20.940007 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:38:20.940605 (+ 598us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:38:20.940608 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:38:20.940721 (+ 113us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:38:20.941282 (+ 561us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 15:38:20.941294 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0809 15:38:20.942831 (+ 1537us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 15:38:20.944951 (+ 2120us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 15:38:20.944961 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 15:38:20.944966 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0809 15:38:20.945281 (+ 315us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 15:38:20.945623 (+ 342us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 15:38:20.945626 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 15:38:20.945628 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 15:38:20.945717 (+ 89us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 15:38:20.946284 (+ 567us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 15:38:20.946290 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 15:38:20.946417 (+ 127us) client_negotiation.cc:769] Sending connection context
0809 15:38:20.946558 (+ 141us) client_negotiation.cc:240] Negotiation successful
0809 15:38:20.946743 (+ 185us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":584,"thread_start_us":110,"threads_started":1}
I20260809 15:38:20.946941 18988 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:38:20.935132 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56888 (local address 127.3.89.193:44707)
0809 15:38:20.935496 (+ 364us) server_negotiation.cc:206] Beginning negotiation
0809 15:38:20.935510 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0809 15:38:20.935919 (+ 409us) server_negotiation.cc:407] Connection header received
0809 15:38:20.936111 (+ 192us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 15:38:20.936116 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 15:38:20.936307 (+ 191us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 15:38:20.936495 (+ 188us) server_negotiation.cc:226] Negotiated authn=SASL
0809 15:38:20.938139 (+ 1644us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:38:20.939106 (+ 967us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:38:20.940123 (+ 1017us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:38:20.940477 (+ 354us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:38:20.940603 (+ 126us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:38:20.942993 (+ 2390us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 15:38:20.943003 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 15:38:20.943014 (+ 11us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 15:38:20.943040 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0809 15:38:20.944833 (+ 1793us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 15:38:20.945412 (+ 579us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 15:38:20.945418 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 15:38:20.945420 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 15:38:20.945505 (+ 85us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 15:38:20.945821 (+ 316us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 15:38:20.945841 (+ 20us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 15:38:20.945842 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 15:38:20.946140 (+ 298us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 15:38:20.946306 (+ 166us) server_negotiation.cc:1035] Waiting for connection context
0809 15:38:20.946635 (+ 329us) server_negotiation.cc:299] Negotiation successful
0809 15:38:20.946788 (+ 153us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":259,"thread_start_us":103,"threads_started":1}
I20260809 15:38:20.948035 18903 tablet_service.cc:1514] Processing DeleteTablet for tablet 2c3257975f264a548a7b27328a081a4e with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 15:38:20 UTC) from {username='kudu', principal='kudu/127.3.89.254@KRBTEST.COM'} at 127.0.0.1:56888
I20260809 15:38:20.948863 18989 tablet_replica.cc:330] stopping tablet replica
I20260809 15:38:20.949162 18989 raft_consensus.cc:2229] T 2c3257975f264a548a7b27328a081a4e P b0a56d93c098468296a2000c9c5609ca [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 15:38:20.949407 18989 raft_consensus.cc:2258] T 2c3257975f264a548a7b27328a081a4e P b0a56d93c098468296a2000c9c5609ca [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 15:38:20.950311 18989 ts_tablet_manager.cc:1905] T 2c3257975f264a548a7b27328a081a4e P b0a56d93c098468296a2000c9c5609ca: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 15:38:20.953871 18989 ts_tablet_manager.cc:1918] T 2c3257975f264a548a7b27328a081a4e P b0a56d93c098468296a2000c9c5609ca: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 15:38:20.953990 18989 log.cc:1196] T 2c3257975f264a548a7b27328a081a4e P b0a56d93c098468296a2000c9c5609ca: Deleting WAL directory at /tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786289732823074-3431-0/minicluster-data/ts-0/wal/wals/2c3257975f264a548a7b27328a081a4e
I20260809 15:38:20.954280 18989 ts_tablet_manager.cc:1939] T 2c3257975f264a548a7b27328a081a4e P b0a56d93c098468296a2000c9c5609ca: Deleting consensus metadata
I20260809 15:38:20.955861 18762 catalog_manager.cc:4841] TS b0a56d93c098468296a2000c9c5609ca (127.3.89.193:44707): tablet 2c3257975f264a548a7b27328a081a4e (table test-table [id=d23e7a7f86ab42efad4fee2ea8b0a852]) successfully deleted
W20260809 15:38:21.029330 18984 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 15:38:21.029587 18984 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 15:38:21.029664 18984 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 15:38:21.033556 18984 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 15:38:21.033641 18984 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 15:38:21.033730 18984 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.3.89.194
I20260809 15:38:21.039067 18984 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.3.89.212:43771
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786289732823074-3431-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786289732823074-3431-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786289732823074-3431-0/krb5kdc/kudu_127.3.89.194.keytab
--principal=kudu/127.3.89.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.3.89.194:45639
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786289732823074-3431-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.3.89.194
--webserver_port=40495
--tserver_master_addrs=127.3.89.254:33497
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.18984
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.3.89.194
--log_dir=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786289732823074-3431-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 09 Aug 2026 14:46:55 UTC on ccycloud.quasar-yswkmf.root.comops.site
build id 4576540
I20260809 15:38:21.040012 18984 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 15:38:21.040966 18984 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 15:38:21.051270 18995 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 15:38:21.051280 18997 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 15:38:21.051281 18994 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 15:38:21.052112 18984 server_base.cc:1034] running on GCE node
I20260809 15:38:21.052573 18984 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 15:38:21.053333 18984 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 15:38:21.054517 18984 hybrid_clock.cc:648] HybridClock initialized: now 1786289901054490 us; error 46 us; skew 500 ppm
Aug 09 15:38:21 dist-test-slave-2w3w krb5kdc[18158](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786289901, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.89.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 15:38:21.057484 18984 init.cc:377] Logged in from keytab as kudu/127.3.89.194@KRBTEST.COM (short username kudu)
I20260809 15:38:21.058313 18984 webserver.cc:460] Webserver started at http://127.3.89.194:40495/ using document root <none> and password file <none>
I20260809 15:38:21.058871 18984 fs_manager.cc:359] Metadata directory not provided
I20260809 15:38:21.058950 18984 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 15:38:21.062590 18984 fs_manager.cc:709] Time spent opening directory manager: real 0.002s user 0.000s sys 0.003s
I20260809 15:38:21.064571 19004 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 15:38:21.065763 18984 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260809 15:38:21.065901 18984 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786289732823074-3431-0/minicluster-data/ts-1/data,/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786289732823074-3431-0/minicluster-data/ts-1/wal
uuid: "273a5fb47ef740c196b1a55c87ed7466"
format_stamp: "Formatted at 2026-08-09 15:38:19 on dist-test-slave-2w3w"
server_key: "c6c967900852bcc2f6e79e6f877d02d2"
server_key_iv: "9d091753c3ded8f1cc2acd366d1e459d"
server_key_version: "encryptionkey@0"
I20260809 15:38:21.066445 18984 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786289732823074-3431-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786289732823074-3431-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786289732823074-3431-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 15:38:21.076953 18984 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 15:38:21.080211 18984 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 15:38:21.080412 18984 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 15:38:21.080972 18984 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 15:38:21.081748 18984 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 15:38:21.081823 18984 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 15:38:21.081890 18984 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 15:38:21.081950 18984 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 15:38:21.091105 18984 rpc_server.cc:307] RPC server started. Bound to: 127.3.89.194:45639
I20260809 15:38:21.091125 19128 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.3.89.194:45639 every 8 connection(s)
I20260809 15:38:21.092089 18984 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786289732823074-3431-0/minicluster-data/ts-1/data/info.pb
I20260809 15:38:21.099284 3431 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskm314YR/build/debug/bin/kudu as pid 18984
I20260809 15:38:21.099797 3431 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskm314YR/build/debug/bin/kudu
/tmp/dist-test-taskm314YR/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786289732823074-3431-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786289732823074-3431-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786289732823074-3431-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786289732823074-3431-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.3.89.195:40257
--local_ip_for_outbound_sockets=127.3.89.195
--tserver_master_addrs=127.3.89.254:33497
--webserver_port=32809
--webserver_interface=127.3.89.195
--keytab_file=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786289732823074-3431-0/krb5kdc/kudu_127.3.89.195.keytab
--principal=kudu/127.3.89.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.3.89.212:43771
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786289732823074-3431-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786289732823074-3431-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786289732823074-3431-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 09 15:38:21 dist-test-slave-2w3w krb5kdc[18158](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786289901, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.89.194@KRBTEST.COM for kudu/127.3.89.254@KRBTEST.COM
I20260809 15:38:21.105772 18981 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:38:21.093901 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.89.194:47551 (local address 127.3.89.254:33497)
0809 15:38:21.094083 (+ 182us) server_negotiation.cc:206] Beginning negotiation
0809 15:38:21.094088 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 15:38:21.095136 (+ 1048us) server_negotiation.cc:407] Connection header received
0809 15:38:21.095937 (+ 801us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 15:38:21.095939 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 15:38:21.095985 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 15:38:21.096063 (+ 78us) server_negotiation.cc:226] Negotiated authn=SASL
0809 15:38:21.097144 (+ 1081us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:38:21.097671 (+ 527us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:38:21.098754 (+ 1083us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:38:21.099039 (+ 285us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:38:21.099119 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:38:21.102071 (+ 2952us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 15:38:21.102074 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 15:38:21.102076 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 15:38:21.102098 (+ 22us) server_negotiation.cc:922] calling sasl_server_start()
0809 15:38:21.103645 (+ 1547us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 15:38:21.104206 (+ 561us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 15:38:21.104209 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 15:38:21.104211 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 15:38:21.104294 (+ 83us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 15:38:21.104638 (+ 344us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 15:38:21.104643 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 15:38:21.104645 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 15:38:21.104831 (+ 186us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 15:38:21.104940 (+ 109us) server_negotiation.cc:1035] Waiting for connection context
0809 15:38:21.105440 (+ 500us) server_negotiation.cc:299] Negotiation successful
0809 15:38:21.105589 (+ 149us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":73}
I20260809 15:38:21.106557 19131 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:38:21.094279 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.89.254:33497 (local address 127.3.89.194:47551)
0809 15:38:21.094928 (+ 649us) negotiation.cc:107] Waiting for socket to connect
0809 15:38:21.095024 (+ 96us) client_negotiation.cc:174] Beginning negotiation
0809 15:38:21.095756 (+ 732us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 15:38:21.096217 (+ 461us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 15:38:21.096223 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 15:38:21.096553 (+ 330us) client_negotiation.cc:189] Negotiated authn=SASL
0809 15:38:21.097011 (+ 458us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:38:21.097024 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:38:21.097782 (+ 758us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:38:21.097784 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:38:21.098645 (+ 861us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:38:21.098654 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:38:21.099204 (+ 550us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:38:21.099208 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:38:21.099414 (+ 206us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:38:21.100192 (+ 778us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 15:38:21.100216 (+ 24us) client_negotiation.cc:656] calling sasl_client_start()
0809 15:38:21.101893 (+ 1677us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 15:38:21.103813 (+ 1920us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 15:38:21.103818 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 15:38:21.103831 (+ 13us) client_negotiation.cc:760] calling sasl_client_step()
0809 15:38:21.104086 (+ 255us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 15:38:21.104389 (+ 303us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 15:38:21.104393 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 15:38:21.104395 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 15:38:21.104537 (+ 142us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 15:38:21.104940 (+ 403us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 15:38:21.104947 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 15:38:21.105186 (+ 239us) client_negotiation.cc:769] Sending connection context
0809 15:38:21.105415 (+ 229us) client_negotiation.cc:240] Negotiation successful
0809 15:38:21.105767 (+ 352us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":461,"thread_start_us":112,"threads_started":1}
I20260809 15:38:21.107771 19129 heartbeater.cc:347] Connected to a master server at 127.3.89.254:33497
I20260809 15:38:21.108016 19129 heartbeater.cc:464] Registering TS with master...
I20260809 15:38:21.108484 19129 heartbeater.cc:511] Master 127.3.89.254:33497 requested a full tablet report, sending...
I20260809 15:38:21.109543 18776 ts_manager.cc:194] Registered new tserver with Master: 273a5fb47ef740c196b1a55c87ed7466 (127.3.89.194:45639)
I20260809 15:38:21.110095 18776 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.3.89.194@KRBTEST.COM'} at 127.3.89.194:47551
W20260809 15:38:21.210153 19132 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 15:38:21.210420 19132 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 15:38:21.210495 19132 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 15:38:21.214401 19132 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 15:38:21.214483 19132 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 15:38:21.214571 19132 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.3.89.195
I20260809 15:38:21.219719 19132 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.3.89.212:43771
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786289732823074-3431-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786289732823074-3431-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786289732823074-3431-0/krb5kdc/kudu_127.3.89.195.keytab
--principal=kudu/127.3.89.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.3.89.195:40257
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786289732823074-3431-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.3.89.195
--webserver_port=32809
--tserver_master_addrs=127.3.89.254:33497
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.19132
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.3.89.195
--log_dir=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786289732823074-3431-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 09 Aug 2026 14:46:55 UTC on ccycloud.quasar-yswkmf.root.comops.site
build id 4576540
I20260809 15:38:21.220646 19132 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 15:38:21.221586 19132 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 15:38:21.230609 19137 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 15:38:21.230611 19140 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 15:38:21.230674 19138 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 15:38:21.230952 19132 server_base.cc:1034] running on GCE node
I20260809 15:38:21.231348 19132 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 15:38:21.231937 19132 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 15:38:21.233117 19132 hybrid_clock.cc:648] HybridClock initialized: now 1786289901233099 us; error 28 us; skew 500 ppm
Aug 09 15:38:21 dist-test-slave-2w3w krb5kdc[18158](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786289901, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.89.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 15:38:21.236364 19132 init.cc:377] Logged in from keytab as kudu/127.3.89.195@KRBTEST.COM (short username kudu)
I20260809 15:38:21.237215 19132 webserver.cc:460] Webserver started at http://127.3.89.195:32809/ using document root <none> and password file <none>
I20260809 15:38:21.237880 19132 fs_manager.cc:359] Metadata directory not provided
I20260809 15:38:21.237960 19132 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 15:38:21.241863 19132 fs_manager.cc:709] Time spent opening directory manager: real 0.002s user 0.001s sys 0.001s
I20260809 15:38:21.243921 19147 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 15:38:21.245641 19132 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260809 15:38:21.245817 19132 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786289732823074-3431-0/minicluster-data/ts-2/data,/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786289732823074-3431-0/minicluster-data/ts-2/wal
uuid: "421cb337efc747e292ba53a026c44c02"
format_stamp: "Formatted at 2026-08-09 15:38:19 on dist-test-slave-2w3w"
server_key: "eb2f06f9d711076fc80f282898368483"
server_key_iv: "b0314ff48b906d23f3b0478ecc39a789"
server_key_version: "encryptionkey@0"
I20260809 15:38:21.246568 19132 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786289732823074-3431-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786289732823074-3431-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786289732823074-3431-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 15:38:21.270455 19132 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 15:38:21.273409 19132 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 15:38:21.273631 19132 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 15:38:21.274240 19132 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 15:38:21.275213 19132 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 15:38:21.275302 19132 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 15:38:21.275383 19132 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 15:38:21.275424 19132 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 15:38:21.284785 19132 rpc_server.cc:307] RPC server started. Bound to: 127.3.89.195:40257
I20260809 15:38:21.284806 19271 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.3.89.195:40257 every 8 connection(s)
I20260809 15:38:21.285806 19132 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786289732823074-3431-0/minicluster-data/ts-2/data/info.pb
I20260809 15:38:21.287114 3431 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskm314YR/build/debug/bin/kudu as pid 19132
I20260809 15:38:21.296341 18981 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:38:21.287818 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47458 (local address 127.3.89.254:33497)
0809 15:38:21.287970 (+ 152us) server_negotiation.cc:206] Beginning negotiation
0809 15:38:21.287976 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 15:38:21.288212 (+ 236us) server_negotiation.cc:407] Connection header received
0809 15:38:21.288393 (+ 181us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 15:38:21.288396 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 15:38:21.288451 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 15:38:21.288541 (+ 90us) server_negotiation.cc:226] Negotiated authn=SASL
0809 15:38:21.289344 (+ 803us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:38:21.289870 (+ 526us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:38:21.290920 (+ 1050us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:38:21.291199 (+ 279us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:38:21.291317 (+ 118us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:38:21.292838 (+ 1521us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 15:38:21.292840 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 15:38:21.292842 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 15:38:21.292865 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0809 15:38:21.294509 (+ 1644us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 15:38:21.295144 (+ 635us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 15:38:21.295146 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 15:38:21.295147 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 15:38:21.295220 (+ 73us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 15:38:21.295519 (+ 299us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 15:38:21.295524 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 15:38:21.295526 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 15:38:21.295691 (+ 165us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 15:38:21.295905 (+ 214us) server_negotiation.cc:1035] Waiting for connection context
0809 15:38:21.296058 (+ 153us) server_negotiation.cc:299] Negotiation successful
0809 15:38:21.296175 (+ 117us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
Aug 09 15:38:21 dist-test-slave-2w3w krb5kdc[18158](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786289901, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.89.195@KRBTEST.COM for kudu/127.3.89.254@KRBTEST.COM
I20260809 15:38:21.301199 19275 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:38:21.288134 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.89.195:45955 (local address 127.3.89.254:33497)
0809 15:38:21.288363 (+ 229us) server_negotiation.cc:206] Beginning negotiation
0809 15:38:21.288368 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 15:38:21.289030 (+ 662us) server_negotiation.cc:407] Connection header received
0809 15:38:21.290380 (+ 1350us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 15:38:21.290383 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 15:38:21.290430 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 15:38:21.290526 (+ 96us) server_negotiation.cc:226] Negotiated authn=SASL
0809 15:38:21.291800 (+ 1274us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:38:21.292351 (+ 551us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:38:21.294100 (+ 1749us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:38:21.294496 (+ 396us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:38:21.294610 (+ 114us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:38:21.297599 (+ 2989us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 15:38:21.297602 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 15:38:21.297603 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 15:38:21.297626 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0809 15:38:21.299479 (+ 1853us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 15:38:21.299923 (+ 444us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 15:38:21.299926 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 15:38:21.299927 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 15:38:21.300004 (+ 77us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 15:38:21.300318 (+ 314us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 15:38:21.300320 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 15:38:21.300321 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 15:38:21.300483 (+ 162us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 15:38:21.300567 (+ 84us) server_negotiation.cc:1035] Waiting for connection context
0809 15:38:21.300978 (+ 411us) server_negotiation.cc:299] Negotiation successful
0809 15:38:21.301078 (+ 100us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":160,"thread_start_us":62,"threads_started":1}
I20260809 15:38:21.301954 19276 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:38:21.288340 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.89.254:33497 (local address 127.3.89.195:45955)
0809 15:38:21.288860 (+ 520us) negotiation.cc:107] Waiting for socket to connect
0809 15:38:21.288914 (+ 54us) client_negotiation.cc:174] Beginning negotiation
0809 15:38:21.290180 (+ 1266us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 15:38:21.290693 (+ 513us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 15:38:21.290700 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 15:38:21.291182 (+ 482us) client_negotiation.cc:189] Negotiated authn=SASL
0809 15:38:21.291652 (+ 470us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:38:21.291665 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:38:21.292504 (+ 839us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:38:21.292508 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:38:21.293976 (+ 1468us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:38:21.293986 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:38:21.294687 (+ 701us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:38:21.294695 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:38:21.294956 (+ 261us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:38:21.295610 (+ 654us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 15:38:21.295627 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0809 15:38:21.297445 (+ 1818us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 15:38:21.299578 (+ 2133us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 15:38:21.299583 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 15:38:21.299593 (+ 10us) client_negotiation.cc:760] calling sasl_client_step()
0809 15:38:21.299829 (+ 236us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 15:38:21.300087 (+ 258us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 15:38:21.300089 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 15:38:21.300091 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 15:38:21.300214 (+ 123us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 15:38:21.300573 (+ 359us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 15:38:21.300577 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 15:38:21.300773 (+ 196us) client_negotiation.cc:769] Sending connection context
0809 15:38:21.300916 (+ 143us) client_negotiation.cc:240] Negotiation successful
0809 15:38:21.301141 (+ 225us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":304,"thread_start_us":145,"threads_started":1}
I20260809 15:38:21.303154 19272 heartbeater.cc:347] Connected to a master server at 127.3.89.254:33497
I20260809 15:38:21.303373 19272 heartbeater.cc:464] Registering TS with master...
I20260809 15:38:21.303809 19272 heartbeater.cc:511] Master 127.3.89.254:33497 requested a full tablet report, sending...
I20260809 15:38:21.304787 18776 ts_manager.cc:194] Registered new tserver with Master: 421cb337efc747e292ba53a026c44c02 (127.3.89.195:40257)
I20260809 15:38:21.305317 18776 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.3.89.195@KRBTEST.COM'} at 127.3.89.195:45955
I20260809 15:38:21.309784 3431 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260809 15:38:21.313838 19275 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:38:21.310343 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47468 (local address 127.3.89.254:33497)
0809 15:38:21.310482 (+ 139us) server_negotiation.cc:206] Beginning negotiation
0809 15:38:21.310489 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 15:38:21.310633 (+ 144us) server_negotiation.cc:407] Connection header received
0809 15:38:21.310746 (+ 113us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 15:38:21.310749 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 15:38:21.310862 (+ 113us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 15:38:21.310949 (+ 87us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 15:38:21.311190 (+ 241us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:38:21.311703 (+ 513us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:38:21.312553 (+ 850us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:38:21.312795 (+ 242us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:38:21.312880 (+ 85us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:38:21.313138 (+ 258us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 15:38:21.313468 (+ 330us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 15:38:21.313559 (+ 91us) server_negotiation.cc:1035] Waiting for connection context
0809 15:38:21.313620 (+ 61us) server_negotiation.cc:299] Negotiation successful
0809 15:38:21.313669 (+ 49us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":62}
I20260809 15:38:21.314451 18776 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin'} at 127.0.0.1:47468:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260809 15:38:21.317814 18776 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260809 15:38:21.325768 18903 tablet_service.cc:1467] Processing CreateTablet for tablet f875addffcb84c74aab667cbd32c47b6 (DEFAULT_TABLE table=test-table [id=f355e85709db40a29134bea82894c531]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 15:38:21.326479 18903 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f875addffcb84c74aab667cbd32c47b6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 15:38:21.329605 18860 tablet_bootstrap.cc:492] T f875addffcb84c74aab667cbd32c47b6 P b0a56d93c098468296a2000c9c5609ca: Bootstrap starting.
I20260809 15:38:21.330087 18987 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:38:21.324637 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.89.195:40257 (local address 127.0.0.1:33678)
0809 15:38:21.324761 (+ 124us) negotiation.cc:107] Waiting for socket to connect
0809 15:38:21.324811 (+ 50us) client_negotiation.cc:174] Beginning negotiation
0809 15:38:21.324878 (+ 67us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 15:38:21.325654 (+ 776us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 15:38:21.325660 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 15:38:21.325682 (+ 22us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 15:38:21.325766 (+ 84us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:38:21.325772 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:38:21.327561 (+ 1789us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:38:21.327565 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:38:21.328541 (+ 976us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:38:21.328551 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:38:21.329667 (+ 1116us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:38:21.329670 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:38:21.329769 (+ 99us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:38:21.329809 (+ 40us) client_negotiation.cc:769] Sending connection context
0809 15:38:21.329878 (+ 69us) client_negotiation.cc:240] Negotiation successful
0809 15:38:21.329963 (+ 85us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":50,"mutex_wait_us":52}
I20260809 15:38:21.330415 19281 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:38:21.324803 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33678 (local address 127.3.89.195:40257)
0809 15:38:21.325181 (+ 378us) server_negotiation.cc:206] Beginning negotiation
0809 15:38:21.325194 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0809 15:38:21.325217 (+ 23us) server_negotiation.cc:407] Connection header received
0809 15:38:21.325288 (+ 71us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 15:38:21.325294 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 15:38:21.325506 (+ 212us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 15:38:21.325673 (+ 167us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 15:38:21.325893 (+ 220us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:38:21.327392 (+ 1499us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:38:21.328675 (+ 1283us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:38:21.329504 (+ 829us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:38:21.329652 (+ 148us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:38:21.329887 (+ 235us) server_negotiation.cc:1035] Waiting for connection context
0809 15:38:21.329939 (+ 52us) server_negotiation.cc:299] Negotiation successful
0809 15:38:21.330036 (+ 97us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":270,"thread_start_us":169,"threads_started":1}
I20260809 15:38:21.330586 18860 tablet_bootstrap.cc:654] T f875addffcb84c74aab667cbd32c47b6 P b0a56d93c098468296a2000c9c5609ca: Neither blocks nor log segments found. Creating new log.
I20260809 15:38:21.331549 19282 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:38:21.325131 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.89.194:45639 (local address 127.0.0.1:37844)
0809 15:38:21.325365 (+ 234us) negotiation.cc:107] Waiting for socket to connect
0809 15:38:21.325378 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0809 15:38:21.325467 (+ 89us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 15:38:21.326603 (+ 1136us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 15:38:21.326606 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 15:38:21.326621 (+ 15us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 15:38:21.326719 (+ 98us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:38:21.326725 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:38:21.327922 (+ 1197us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:38:21.327926 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:38:21.329412 (+ 1486us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:38:21.329424 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:38:21.331080 (+ 1656us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:38:21.331084 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:38:21.331201 (+ 117us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:38:21.331231 (+ 30us) client_negotiation.cc:769] Sending connection context
0809 15:38:21.331314 (+ 83us) client_negotiation.cc:240] Negotiation successful
0809 15:38:21.331406 (+ 92us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":173,"mutex_wait_us":1,"thread_start_us":107,"threads_started":1}
I20260809 15:38:21.332152 19283 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:38:21.325569 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37844 (local address 127.3.89.194:45639)
0809 15:38:21.325946 (+ 377us) server_negotiation.cc:206] Beginning negotiation
0809 15:38:21.325984 (+ 38us) server_negotiation.cc:399] Waiting for connection header
0809 15:38:21.326005 (+ 21us) server_negotiation.cc:407] Connection header received
0809 15:38:21.326079 (+ 74us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 15:38:21.326083 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 15:38:21.326266 (+ 183us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 15:38:21.326419 (+ 153us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 15:38:21.326884 (+ 465us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:38:21.327768 (+ 884us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:38:21.329722 (+ 1954us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:38:21.330643 (+ 921us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:38:21.330755 (+ 112us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:38:21.330943 (+ 188us) server_negotiation.cc:1035] Waiting for connection context
0809 15:38:21.331865 (+ 922us) server_negotiation.cc:299] Negotiation successful
0809 15:38:21.331979 (+ 114us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":289,"thread_start_us":91,"threads_started":1}
I20260809 15:38:21.332700 18860 tablet_bootstrap.cc:492] T f875addffcb84c74aab667cbd32c47b6 P b0a56d93c098468296a2000c9c5609ca: No bootstrap required, opened a new log
I20260809 15:38:21.332803 18860 ts_tablet_manager.cc:1397] T f875addffcb84c74aab667cbd32c47b6 P b0a56d93c098468296a2000c9c5609ca: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260809 15:38:21.333258 18860 raft_consensus.cc:348] T f875addffcb84c74aab667cbd32c47b6 P b0a56d93c098468296a2000c9c5609ca [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "421cb337efc747e292ba53a026c44c02" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 40257 } } peers { permanent_uuid: "273a5fb47ef740c196b1a55c87ed7466" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 45639 } } peers { permanent_uuid: "b0a56d93c098468296a2000c9c5609ca" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 44707 } }
I20260809 15:38:21.333408 18860 raft_consensus.cc:374] T f875addffcb84c74aab667cbd32c47b6 P b0a56d93c098468296a2000c9c5609ca [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 15:38:21.333465 18860 raft_consensus.cc:729] T f875addffcb84c74aab667cbd32c47b6 P b0a56d93c098468296a2000c9c5609ca [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b0a56d93c098468296a2000c9c5609ca, State: Initialized, Role: FOLLOWER
I20260809 15:38:21.333606 18860 consensus_queue.cc:260] T f875addffcb84c74aab667cbd32c47b6 P b0a56d93c098468296a2000c9c5609ca [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "421cb337efc747e292ba53a026c44c02" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 40257 } } peers { permanent_uuid: "273a5fb47ef740c196b1a55c87ed7466" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 45639 } } peers { permanent_uuid: "b0a56d93c098468296a2000c9c5609ca" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 44707 } }
I20260809 15:38:21.333950 18860 ts_tablet_manager.cc:1428] T f875addffcb84c74aab667cbd32c47b6 P b0a56d93c098468296a2000c9c5609ca: Time spent starting tablet: real 0.001s user 0.002s sys 0.000s
I20260809 15:38:21.334021 19199 tablet_service.cc:1467] Processing CreateTablet for tablet f875addffcb84c74aab667cbd32c47b6 (DEFAULT_TABLE table=test-table [id=f355e85709db40a29134bea82894c531]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 15:38:21.335196 19199 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f875addffcb84c74aab667cbd32c47b6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 15:38:21.334879 19057 tablet_service.cc:1467] Processing CreateTablet for tablet f875addffcb84c74aab667cbd32c47b6 (DEFAULT_TABLE table=test-table [id=f355e85709db40a29134bea82894c531]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 15:38:21.335863 19057 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f875addffcb84c74aab667cbd32c47b6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 15:38:21.343397 19285 tablet_bootstrap.cc:492] T f875addffcb84c74aab667cbd32c47b6 P 273a5fb47ef740c196b1a55c87ed7466: Bootstrap starting.
I20260809 15:38:21.343990 19286 tablet_bootstrap.cc:492] T f875addffcb84c74aab667cbd32c47b6 P 421cb337efc747e292ba53a026c44c02: Bootstrap starting.
I20260809 15:38:21.346510 19285 tablet_bootstrap.cc:654] T f875addffcb84c74aab667cbd32c47b6 P 273a5fb47ef740c196b1a55c87ed7466: Neither blocks nor log segments found. Creating new log.
I20260809 15:38:21.346627 19286 tablet_bootstrap.cc:654] T f875addffcb84c74aab667cbd32c47b6 P 421cb337efc747e292ba53a026c44c02: Neither blocks nor log segments found. Creating new log.
I20260809 15:38:21.347548 19286 log.cc:824] T f875addffcb84c74aab667cbd32c47b6 P 421cb337efc747e292ba53a026c44c02: Log is configured to *not* fsync() on all Append() calls
I20260809 15:38:21.347528 19285 log.cc:824] T f875addffcb84c74aab667cbd32c47b6 P 273a5fb47ef740c196b1a55c87ed7466: Log is configured to *not* fsync() on all Append() calls
I20260809 15:38:21.349407 19286 tablet_bootstrap.cc:492] T f875addffcb84c74aab667cbd32c47b6 P 421cb337efc747e292ba53a026c44c02: No bootstrap required, opened a new log
I20260809 15:38:21.349428 19285 tablet_bootstrap.cc:492] T f875addffcb84c74aab667cbd32c47b6 P 273a5fb47ef740c196b1a55c87ed7466: No bootstrap required, opened a new log
I20260809 15:38:21.349620 19286 ts_tablet_manager.cc:1397] T f875addffcb84c74aab667cbd32c47b6 P 421cb337efc747e292ba53a026c44c02: Time spent bootstrapping tablet: real 0.006s user 0.004s sys 0.000s
I20260809 15:38:21.349650 19285 ts_tablet_manager.cc:1397] T f875addffcb84c74aab667cbd32c47b6 P 273a5fb47ef740c196b1a55c87ed7466: Time spent bootstrapping tablet: real 0.006s user 0.005s sys 0.000s
I20260809 15:38:21.352892 19286 raft_consensus.cc:348] T f875addffcb84c74aab667cbd32c47b6 P 421cb337efc747e292ba53a026c44c02 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "421cb337efc747e292ba53a026c44c02" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 40257 } } peers { permanent_uuid: "273a5fb47ef740c196b1a55c87ed7466" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 45639 } } peers { permanent_uuid: "b0a56d93c098468296a2000c9c5609ca" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 44707 } }
I20260809 15:38:21.352893 19285 raft_consensus.cc:348] T f875addffcb84c74aab667cbd32c47b6 P 273a5fb47ef740c196b1a55c87ed7466 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "421cb337efc747e292ba53a026c44c02" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 40257 } } peers { permanent_uuid: "273a5fb47ef740c196b1a55c87ed7466" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 45639 } } peers { permanent_uuid: "b0a56d93c098468296a2000c9c5609ca" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 44707 } }
I20260809 15:38:21.353137 19286 raft_consensus.cc:374] T f875addffcb84c74aab667cbd32c47b6 P 421cb337efc747e292ba53a026c44c02 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 15:38:21.353132 19285 raft_consensus.cc:374] T f875addffcb84c74aab667cbd32c47b6 P 273a5fb47ef740c196b1a55c87ed7466 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 15:38:21.353200 19286 raft_consensus.cc:729] T f875addffcb84c74aab667cbd32c47b6 P 421cb337efc747e292ba53a026c44c02 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 421cb337efc747e292ba53a026c44c02, State: Initialized, Role: FOLLOWER
I20260809 15:38:21.353221 19285 raft_consensus.cc:729] T f875addffcb84c74aab667cbd32c47b6 P 273a5fb47ef740c196b1a55c87ed7466 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 273a5fb47ef740c196b1a55c87ed7466, State: Initialized, Role: FOLLOWER
I20260809 15:38:21.353690 19285 consensus_queue.cc:260] T f875addffcb84c74aab667cbd32c47b6 P 273a5fb47ef740c196b1a55c87ed7466 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "421cb337efc747e292ba53a026c44c02" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 40257 } } peers { permanent_uuid: "273a5fb47ef740c196b1a55c87ed7466" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 45639 } } peers { permanent_uuid: "b0a56d93c098468296a2000c9c5609ca" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 44707 } }
I20260809 15:38:21.353682 19286 consensus_queue.cc:260] T f875addffcb84c74aab667cbd32c47b6 P 421cb337efc747e292ba53a026c44c02 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "421cb337efc747e292ba53a026c44c02" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 40257 } } peers { permanent_uuid: "273a5fb47ef740c196b1a55c87ed7466" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 45639 } } peers { permanent_uuid: "b0a56d93c098468296a2000c9c5609ca" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 44707 } }
I20260809 15:38:21.354423 19272 heartbeater.cc:503] Master 127.3.89.254:33497 was elected leader, sending a full tablet report...
I20260809 15:38:21.354743 19286 ts_tablet_manager.cc:1428] T f875addffcb84c74aab667cbd32c47b6 P 421cb337efc747e292ba53a026c44c02: Time spent starting tablet: real 0.005s user 0.006s sys 0.000s
I20260809 15:38:21.354760 19285 ts_tablet_manager.cc:1428] T f875addffcb84c74aab667cbd32c47b6 P 273a5fb47ef740c196b1a55c87ed7466: Time spent starting tablet: real 0.005s user 0.006s sys 0.000s
I20260809 15:38:21.355145 19129 heartbeater.cc:503] Master 127.3.89.254:33497 was elected leader, sending a full tablet report...
I20260809 15:38:21.403250 19290 raft_consensus.cc:482] T f875addffcb84c74aab667cbd32c47b6 P 273a5fb47ef740c196b1a55c87ed7466 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 15:38:21.403475 19290 raft_consensus.cc:504] T f875addffcb84c74aab667cbd32c47b6 P 273a5fb47ef740c196b1a55c87ed7466 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "421cb337efc747e292ba53a026c44c02" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 40257 } } peers { permanent_uuid: "273a5fb47ef740c196b1a55c87ed7466" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 45639 } } peers { permanent_uuid: "b0a56d93c098468296a2000c9c5609ca" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 44707 } }
I20260809 15:38:21.404670 19290 leader_election.cc:288] T f875addffcb84c74aab667cbd32c47b6 P 273a5fb47ef740c196b1a55c87ed7466 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 421cb337efc747e292ba53a026c44c02 (127.3.89.195:40257), b0a56d93c098468296a2000c9c5609ca (127.3.89.193:44707)
I20260809 15:38:21.406410 18986 raft_consensus.cc:482] T f875addffcb84c74aab667cbd32c47b6 P b0a56d93c098468296a2000c9c5609ca [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 15:38:21.406599 18986 raft_consensus.cc:504] T f875addffcb84c74aab667cbd32c47b6 P b0a56d93c098468296a2000c9c5609ca [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "421cb337efc747e292ba53a026c44c02" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 40257 } } peers { permanent_uuid: "273a5fb47ef740c196b1a55c87ed7466" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 45639 } } peers { permanent_uuid: "b0a56d93c098468296a2000c9c5609ca" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 44707 } }
I20260809 15:38:21.407732 18986 leader_election.cc:288] T f875addffcb84c74aab667cbd32c47b6 P b0a56d93c098468296a2000c9c5609ca [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 421cb337efc747e292ba53a026c44c02 (127.3.89.195:40257), 273a5fb47ef740c196b1a55c87ed7466 (127.3.89.194:45639)
W20260809 15:38:21.408133 18980 tablet.cc:2369] T f875addffcb84c74aab667cbd32c47b6 P b0a56d93c098468296a2000c9c5609ca: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260809 15:38:21.409492 19281 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:38:21.405012 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.89.194:58185 (local address 127.3.89.195:40257)
0809 15:38:21.405170 (+ 158us) server_negotiation.cc:206] Beginning negotiation
0809 15:38:21.405175 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 15:38:21.405449 (+ 274us) server_negotiation.cc:407] Connection header received
0809 15:38:21.405619 (+ 170us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 15:38:21.405624 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 15:38:21.405674 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 15:38:21.405761 (+ 87us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 15:38:21.406029 (+ 268us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:38:21.406685 (+ 656us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:38:21.408175 (+ 1490us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:38:21.408768 (+ 593us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:38:21.408855 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:38:21.408925 (+ 70us) server_negotiation.cc:1035] Waiting for connection context
0809 15:38:21.409291 (+ 366us) server_negotiation.cc:299] Negotiation successful
0809 15:38:21.409362 (+ 71us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":52}
I20260809 15:38:21.409914 19291 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:38:21.404918 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.89.195:40257 (local address 127.3.89.194:58185)
0809 15:38:21.405352 (+ 434us) negotiation.cc:107] Waiting for socket to connect
0809 15:38:21.405364 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0809 15:38:21.405505 (+ 141us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 15:38:21.405774 (+ 269us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 15:38:21.405776 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 15:38:21.405792 (+ 16us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 15:38:21.405916 (+ 124us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:38:21.405922 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:38:21.406863 (+ 941us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:38:21.406866 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:38:21.408037 (+ 1171us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:38:21.408049 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:38:21.409031 (+ 982us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:38:21.409036 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:38:21.409145 (+ 109us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:38:21.409169 (+ 24us) client_negotiation.cc:769] Sending connection context
0809 15:38:21.409683 (+ 514us) client_negotiation.cc:240] Negotiation successful
0809 15:38:21.409766 (+ 83us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":381,"mutex_wait_us":59,"thread_start_us":144,"threads_started":1}
I20260809 15:38:21.410086 19131 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:38:21.404918 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.89.193:44707 (local address 127.3.89.194:46393)
0809 15:38:21.405034 (+ 116us) negotiation.cc:107] Waiting for socket to connect
0809 15:38:21.405052 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0809 15:38:21.405151 (+ 99us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 15:38:21.405575 (+ 424us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 15:38:21.405579 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 15:38:21.405597 (+ 18us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 15:38:21.405717 (+ 120us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:38:21.405722 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:38:21.406632 (+ 910us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:38:21.406635 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:38:21.407878 (+ 1243us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:38:21.407890 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:38:21.409597 (+ 1707us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:38:21.409601 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:38:21.409735 (+ 134us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:38:21.409824 (+ 89us) client_negotiation.cc:769] Sending connection context
0809 15:38:21.409892 (+ 68us) client_negotiation.cc:240] Negotiation successful
0809 15:38:21.409964 (+ 72us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":30,"mutex_wait_us":43}
I20260809 15:38:21.410540 18988 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:38:21.405209 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.89.194:46393 (local address 127.3.89.193:44707)
0809 15:38:21.405330 (+ 121us) server_negotiation.cc:206] Beginning negotiation
0809 15:38:21.405333 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0809 15:38:21.405344 (+ 11us) server_negotiation.cc:407] Connection header received
0809 15:38:21.405391 (+ 47us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 15:38:21.405393 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 15:38:21.405441 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 15:38:21.405557 (+ 116us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 15:38:21.405840 (+ 283us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:38:21.406479 (+ 639us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:38:21.408622 (+ 2143us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:38:21.409285 (+ 663us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:38:21.409393 (+ 108us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:38:21.409534 (+ 141us) server_negotiation.cc:1035] Waiting for connection context
0809 15:38:21.410332 (+ 798us) server_negotiation.cc:299] Negotiation successful
0809 15:38:21.410405 (+ 73us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":56}
I20260809 15:38:21.410502 19222 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f875addffcb84c74aab667cbd32c47b6" candidate_uuid: "273a5fb47ef740c196b1a55c87ed7466" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "421cb337efc747e292ba53a026c44c02" is_pre_election: true
I20260809 15:38:21.410784 19222 raft_consensus.cc:2454] T f875addffcb84c74aab667cbd32c47b6 P 421cb337efc747e292ba53a026c44c02 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 273a5fb47ef740c196b1a55c87ed7466 in term 0.
I20260809 15:38:21.411008 18926 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f875addffcb84c74aab667cbd32c47b6" candidate_uuid: "273a5fb47ef740c196b1a55c87ed7466" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b0a56d93c098468296a2000c9c5609ca" is_pre_election: true
I20260809 15:38:21.411221 18926 raft_consensus.cc:2454] T f875addffcb84c74aab667cbd32c47b6 P b0a56d93c098468296a2000c9c5609ca [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 273a5fb47ef740c196b1a55c87ed7466 in term 0.
I20260809 15:38:21.411271 19005 leader_election.cc:302] T f875addffcb84c74aab667cbd32c47b6 P 273a5fb47ef740c196b1a55c87ed7466 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 273a5fb47ef740c196b1a55c87ed7466, 421cb337efc747e292ba53a026c44c02; no voters:
I20260809 15:38:21.411810 19290 raft_consensus.cc:2781] T f875addffcb84c74aab667cbd32c47b6 P 273a5fb47ef740c196b1a55c87ed7466 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260809 15:38:21.411934 19290 raft_consensus.cc:482] T f875addffcb84c74aab667cbd32c47b6 P 273a5fb47ef740c196b1a55c87ed7466 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 15:38:21.411998 19290 raft_consensus.cc:3037] T f875addffcb84c74aab667cbd32c47b6 P 273a5fb47ef740c196b1a55c87ed7466 [term 0 FOLLOWER]: Advancing to term 1
I20260809 15:38:21.413295 18982 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:38:21.407892 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.89.195:40257 (local address 127.3.89.193:50329)
0809 15:38:21.408041 (+ 149us) negotiation.cc:107] Waiting for socket to connect
0809 15:38:21.408054 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0809 15:38:21.408134 (+ 80us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 15:38:21.408707 (+ 573us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 15:38:21.408712 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 15:38:21.408730 (+ 18us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 15:38:21.408848 (+ 118us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:38:21.408855 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:38:21.410426 (+ 1571us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:38:21.410429 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:38:21.411745 (+ 1316us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:38:21.411759 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:38:21.412850 (+ 1091us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:38:21.412857 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:38:21.412969 (+ 112us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:38:21.412994 (+ 25us) client_negotiation.cc:769] Sending connection context
0809 15:38:21.413082 (+ 88us) client_negotiation.cc:240] Negotiation successful
0809 15:38:21.413155 (+ 73us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":89,"mutex_wait_us":68}
I20260809 15:38:21.413393 19283 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:38:21.408691 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.89.193:46113 (local address 127.3.89.194:45639)
0809 15:38:21.408803 (+ 112us) server_negotiation.cc:206] Beginning negotiation
0809 15:38:21.408807 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 15:38:21.409045 (+ 238us) server_negotiation.cc:407] Connection header received
0809 15:38:21.409208 (+ 163us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 15:38:21.409212 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 15:38:21.409256 (+ 44us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 15:38:21.409330 (+ 74us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 15:38:21.409699 (+ 369us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:38:21.410414 (+ 715us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:38:21.412185 (+ 1771us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:38:21.412824 (+ 639us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:38:21.412927 (+ 103us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:38:21.412974 (+ 47us) server_negotiation.cc:1035] Waiting for connection context
0809 15:38:21.413211 (+ 237us) server_negotiation.cc:299] Negotiation successful
0809 15:38:21.413277 (+ 66us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":22,"server-negotiator.queue_time_us":39}
I20260809 15:38:21.413314 19292 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:38:21.407972 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.89.193:50329 (local address 127.3.89.195:40257)
0809 15:38:21.408445 (+ 473us) server_negotiation.cc:206] Beginning negotiation
0809 15:38:21.408453 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0809 15:38:21.408478 (+ 25us) server_negotiation.cc:407] Connection header received
0809 15:38:21.408515 (+ 37us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 15:38:21.408517 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 15:38:21.408556 (+ 39us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 15:38:21.408948 (+ 392us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 15:38:21.409056 (+ 108us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:38:21.410302 (+ 1246us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:38:21.411910 (+ 1608us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:38:21.412591 (+ 681us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:38:21.412710 (+ 119us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:38:21.412752 (+ 42us) server_negotiation.cc:1035] Waiting for connection context
0809 15:38:21.413096 (+ 344us) server_negotiation.cc:299] Negotiation successful
0809 15:38:21.413185 (+ 89us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":408,"thread_start_us":150,"threads_started":1}
I20260809 15:38:21.413661 19290 raft_consensus.cc:504] T f875addffcb84c74aab667cbd32c47b6 P 273a5fb47ef740c196b1a55c87ed7466 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "421cb337efc747e292ba53a026c44c02" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 40257 } } peers { permanent_uuid: "273a5fb47ef740c196b1a55c87ed7466" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 45639 } } peers { permanent_uuid: "b0a56d93c098468296a2000c9c5609ca" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 44707 } }
I20260809 15:38:21.413766 19293 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:38:21.408411 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.89.194:45639 (local address 127.3.89.193:46113)
0809 15:38:21.408959 (+ 548us) negotiation.cc:107] Waiting for socket to connect
0809 15:38:21.408973 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0809 15:38:21.409102 (+ 129us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 15:38:21.409471 (+ 369us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 15:38:21.409473 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 15:38:21.409491 (+ 18us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 15:38:21.409580 (+ 89us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:38:21.409586 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:38:21.410534 (+ 948us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:38:21.410537 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:38:21.411967 (+ 1430us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:38:21.411977 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:38:21.412953 (+ 976us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:38:21.412965 (+ 12us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:38:21.413065 (+ 100us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:38:21.413091 (+ 26us) client_negotiation.cc:769] Sending connection context
0809 15:38:21.413517 (+ 426us) client_negotiation.cc:240] Negotiation successful
0809 15:38:21.413602 (+ 85us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":490,"mutex_wait_us":6,"thread_start_us":68,"threads_started":1}
I20260809 15:38:21.413946 19222 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f875addffcb84c74aab667cbd32c47b6" candidate_uuid: "b0a56d93c098468296a2000c9c5609ca" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "421cb337efc747e292ba53a026c44c02" is_pre_election: true
I20260809 15:38:21.414063 19222 raft_consensus.cc:2454] T f875addffcb84c74aab667cbd32c47b6 P 421cb337efc747e292ba53a026c44c02 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b0a56d93c098468296a2000c9c5609ca in term 0.
I20260809 15:38:21.414412 18854 leader_election.cc:302] T f875addffcb84c74aab667cbd32c47b6 P b0a56d93c098468296a2000c9c5609ca [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 421cb337efc747e292ba53a026c44c02, b0a56d93c098468296a2000c9c5609ca; no voters:
I20260809 15:38:21.414422 19078 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f875addffcb84c74aab667cbd32c47b6" candidate_uuid: "b0a56d93c098468296a2000c9c5609ca" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "273a5fb47ef740c196b1a55c87ed7466" is_pre_election: true
I20260809 15:38:21.414664 19078 raft_consensus.cc:2379] T f875addffcb84c74aab667cbd32c47b6 P 273a5fb47ef740c196b1a55c87ed7466 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b0a56d93c098468296a2000c9c5609ca in current term 1: Already voted for candidate 273a5fb47ef740c196b1a55c87ed7466 in this term.
I20260809 15:38:21.414700 18986 raft_consensus.cc:2781] T f875addffcb84c74aab667cbd32c47b6 P b0a56d93c098468296a2000c9c5609ca [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260809 15:38:21.414813 18986 raft_consensus.cc:482] T f875addffcb84c74aab667cbd32c47b6 P b0a56d93c098468296a2000c9c5609ca [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 15:38:21.414814 19222 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f875addffcb84c74aab667cbd32c47b6" candidate_uuid: "273a5fb47ef740c196b1a55c87ed7466" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "421cb337efc747e292ba53a026c44c02"
I20260809 15:38:21.414871 18986 raft_consensus.cc:3037] T f875addffcb84c74aab667cbd32c47b6 P b0a56d93c098468296a2000c9c5609ca [term 0 FOLLOWER]: Advancing to term 1
I20260809 15:38:21.414925 19222 raft_consensus.cc:3037] T f875addffcb84c74aab667cbd32c47b6 P 421cb337efc747e292ba53a026c44c02 [term 0 FOLLOWER]: Advancing to term 1
I20260809 15:38:21.415506 18926 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f875addffcb84c74aab667cbd32c47b6" candidate_uuid: "273a5fb47ef740c196b1a55c87ed7466" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b0a56d93c098468296a2000c9c5609ca"
I20260809 15:38:21.415747 19290 leader_election.cc:288] T f875addffcb84c74aab667cbd32c47b6 P 273a5fb47ef740c196b1a55c87ed7466 [CANDIDATE]: Term 1 election: Requested vote from peers 421cb337efc747e292ba53a026c44c02 (127.3.89.195:40257), b0a56d93c098468296a2000c9c5609ca (127.3.89.193:44707)
I20260809 15:38:21.416221 18986 raft_consensus.cc:504] T f875addffcb84c74aab667cbd32c47b6 P b0a56d93c098468296a2000c9c5609ca [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "421cb337efc747e292ba53a026c44c02" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 40257 } } peers { permanent_uuid: "273a5fb47ef740c196b1a55c87ed7466" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 45639 } } peers { permanent_uuid: "b0a56d93c098468296a2000c9c5609ca" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 44707 } }
I20260809 15:38:21.416267 19222 raft_consensus.cc:2454] T f875addffcb84c74aab667cbd32c47b6 P 421cb337efc747e292ba53a026c44c02 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 273a5fb47ef740c196b1a55c87ed7466 in term 1.
I20260809 15:38:21.416465 18926 raft_consensus.cc:2379] T f875addffcb84c74aab667cbd32c47b6 P b0a56d93c098468296a2000c9c5609ca [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate 273a5fb47ef740c196b1a55c87ed7466 in current term 1: Already voted for candidate b0a56d93c098468296a2000c9c5609ca in this term.
I20260809 15:38:21.416579 19005 leader_election.cc:302] T f875addffcb84c74aab667cbd32c47b6 P 273a5fb47ef740c196b1a55c87ed7466 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 273a5fb47ef740c196b1a55c87ed7466, 421cb337efc747e292ba53a026c44c02; no voters:
I20260809 15:38:21.416623 18986 leader_election.cc:288] T f875addffcb84c74aab667cbd32c47b6 P b0a56d93c098468296a2000c9c5609ca [CANDIDATE]: Term 1 election: Requested vote from peers 421cb337efc747e292ba53a026c44c02 (127.3.89.195:40257), 273a5fb47ef740c196b1a55c87ed7466 (127.3.89.194:45639)
I20260809 15:38:21.416905 19222 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f875addffcb84c74aab667cbd32c47b6" candidate_uuid: "b0a56d93c098468296a2000c9c5609ca" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "421cb337efc747e292ba53a026c44c02"
I20260809 15:38:21.417058 19078 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f875addffcb84c74aab667cbd32c47b6" candidate_uuid: "b0a56d93c098468296a2000c9c5609ca" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "273a5fb47ef740c196b1a55c87ed7466"
I20260809 15:38:21.417104 19222 raft_consensus.cc:2379] T f875addffcb84c74aab667cbd32c47b6 P 421cb337efc747e292ba53a026c44c02 [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate b0a56d93c098468296a2000c9c5609ca in current term 1: Already voted for candidate 273a5fb47ef740c196b1a55c87ed7466 in this term.
I20260809 15:38:21.417140 19290 raft_consensus.cc:2781] T f875addffcb84c74aab667cbd32c47b6 P 273a5fb47ef740c196b1a55c87ed7466 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 15:38:21.417354 19290 raft_consensus.cc:686] T f875addffcb84c74aab667cbd32c47b6 P 273a5fb47ef740c196b1a55c87ed7466 [term 1 LEADER]: Becoming Leader. State: Replica: 273a5fb47ef740c196b1a55c87ed7466, State: Running, Role: LEADER
I20260809 15:38:21.417691 19290 consensus_queue.cc:237] T f875addffcb84c74aab667cbd32c47b6 P 273a5fb47ef740c196b1a55c87ed7466 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "421cb337efc747e292ba53a026c44c02" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 40257 } } peers { permanent_uuid: "273a5fb47ef740c196b1a55c87ed7466" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 45639 } } peers { permanent_uuid: "b0a56d93c098468296a2000c9c5609ca" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 44707 } }
I20260809 15:38:21.420269 18856 leader_election.cc:302] T f875addffcb84c74aab667cbd32c47b6 P b0a56d93c098468296a2000c9c5609ca [CANDIDATE]: Term 1 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b0a56d93c098468296a2000c9c5609ca; no voters: 273a5fb47ef740c196b1a55c87ed7466, 421cb337efc747e292ba53a026c44c02
I20260809 15:38:21.420517 18986 raft_consensus.cc:2726] T f875addffcb84c74aab667cbd32c47b6 P b0a56d93c098468296a2000c9c5609ca [term 1 FOLLOWER]: Leader election lost for term 1. Reason: could not achieve majority
I20260809 15:38:21.421168 18776 catalog_manager.cc:5495] T f875addffcb84c74aab667cbd32c47b6 P 273a5fb47ef740c196b1a55c87ed7466 reported cstate change: term changed from 0 to 1, leader changed from <none> to 273a5fb47ef740c196b1a55c87ed7466 (127.3.89.194). New cstate: current_term: 1 leader_uuid: "273a5fb47ef740c196b1a55c87ed7466" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "421cb337efc747e292ba53a026c44c02" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 40257 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "273a5fb47ef740c196b1a55c87ed7466" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 45639 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "b0a56d93c098468296a2000c9c5609ca" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 44707 } health_report { overall_health: UNKNOWN } } }
I20260809 15:38:21.434389 19283 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:38:21.431099 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37856 (local address 127.3.89.194:45639)
0809 15:38:21.431301 (+ 202us) server_negotiation.cc:206] Beginning negotiation
0809 15:38:21.431305 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 15:38:21.431325 (+ 20us) server_negotiation.cc:407] Connection header received
0809 15:38:21.431389 (+ 64us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 15:38:21.431391 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 15:38:21.431439 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 15:38:21.431528 (+ 89us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 15:38:21.431740 (+ 212us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:38:21.432228 (+ 488us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:38:21.433057 (+ 829us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:38:21.433259 (+ 202us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:38:21.433333 (+ 74us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:38:21.433595 (+ 262us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 15:38:21.433985 (+ 390us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 15:38:21.434074 (+ 89us) server_negotiation.cc:1035] Waiting for connection context
0809 15:38:21.434152 (+ 78us) server_negotiation.cc:299] Negotiation successful
0809 15:38:21.434197 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":61}
I20260809 15:38:21.440212 18926 raft_consensus.cc:1261] T f875addffcb84c74aab667cbd32c47b6 P b0a56d93c098468296a2000c9c5609ca [term 1 FOLLOWER]: Refusing update from remote peer 273a5fb47ef740c196b1a55c87ed7466: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 15:38:21.440228 19222 raft_consensus.cc:1261] T f875addffcb84c74aab667cbd32c47b6 P 421cb337efc747e292ba53a026c44c02 [term 1 FOLLOWER]: Refusing update from remote peer 273a5fb47ef740c196b1a55c87ed7466: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 15:38:21.440770 19290 consensus_queue.cc:1035] T f875addffcb84c74aab667cbd32c47b6 P 273a5fb47ef740c196b1a55c87ed7466 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b0a56d93c098468296a2000c9c5609ca" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 44707 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 15:38:21.440932 19294 consensus_queue.cc:1035] T f875addffcb84c74aab667cbd32c47b6 P 273a5fb47ef740c196b1a55c87ed7466 [LEADER]: Connected to new peer: Peer: permanent_uuid: "421cb337efc747e292ba53a026c44c02" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 40257 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 15:38:21.447481 19296 mvcc.cc:205] Tried to move back new op lower bound from 7316643436292063232 to 7316643436209029120. Current Snapshot: MvccSnapshot[applied={T|T < 7316643436292063232}]
I20260809 15:38:21.456882 18776 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin'} at 127.0.0.1:47468:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 15:38:21.457069 18776 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin'} at 127.0.0.1:47468:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 15:38:21.459424 18776 catalog_manager.cc:5782] T 00000000000000000000000000000000 P f5ecfca57914460d8be340f7b52004b5: Sending DeleteTablet for 3 replicas of tablet f875addffcb84c74aab667cbd32c47b6
I20260809 15:38:21.460094 19199 tablet_service.cc:1514] Processing DeleteTablet for tablet f875addffcb84c74aab667cbd32c47b6 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 15:38:21 UTC) from {username='kudu', principal='kudu/127.3.89.254@KRBTEST.COM'} at 127.0.0.1:33678
I20260809 15:38:21.460165 19057 tablet_service.cc:1514] Processing DeleteTablet for tablet f875addffcb84c74aab667cbd32c47b6 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 15:38:21 UTC) from {username='kudu', principal='kudu/127.3.89.254@KRBTEST.COM'} at 127.0.0.1:37844
I20260809 15:38:21.460565 19306 tablet_replica.cc:330] T f875addffcb84c74aab667cbd32c47b6 P 421cb337efc747e292ba53a026c44c02: stopping tablet replica
I20260809 15:38:21.460579 19307 tablet_replica.cc:330] T f875addffcb84c74aab667cbd32c47b6 P 273a5fb47ef740c196b1a55c87ed7466: stopping tablet replica
I20260809 15:38:21.460690 18903 tablet_service.cc:1514] Processing DeleteTablet for tablet f875addffcb84c74aab667cbd32c47b6 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 15:38:21 UTC) from {username='kudu', principal='kudu/127.3.89.254@KRBTEST.COM'} at 127.0.0.1:56888
I20260809 15:38:21.460866 19307 raft_consensus.cc:2229] T f875addffcb84c74aab667cbd32c47b6 P 273a5fb47ef740c196b1a55c87ed7466 [term 1 LEADER]: Raft consensus shutting down.
I20260809 15:38:21.460922 19306 raft_consensus.cc:2229] T f875addffcb84c74aab667cbd32c47b6 P 421cb337efc747e292ba53a026c44c02 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 15:38:21.461216 19307 raft_consensus.cc:2258] T f875addffcb84c74aab667cbd32c47b6 P 273a5fb47ef740c196b1a55c87ed7466 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 15:38:21.461272 3431 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskm314YR/build/debug/bin/kudu with pid 18815
I20260809 15:38:21.461261 19306 raft_consensus.cc:2258] T f875addffcb84c74aab667cbd32c47b6 P 421cb337efc747e292ba53a026c44c02 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 15:38:21.464504 19307 ts_tablet_manager.cc:1905] T f875addffcb84c74aab667cbd32c47b6 P 273a5fb47ef740c196b1a55c87ed7466: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 15:38:21.464504 19306 ts_tablet_manager.cc:1905] T f875addffcb84c74aab667cbd32c47b6 P 421cb337efc747e292ba53a026c44c02: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 15:38:21.467200 19306 ts_tablet_manager.cc:1918] T f875addffcb84c74aab667cbd32c47b6 P 421cb337efc747e292ba53a026c44c02: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 15:38:21.467200 19307 ts_tablet_manager.cc:1918] T f875addffcb84c74aab667cbd32c47b6 P 273a5fb47ef740c196b1a55c87ed7466: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 15:38:21.467298 19307 log.cc:1196] T f875addffcb84c74aab667cbd32c47b6 P 273a5fb47ef740c196b1a55c87ed7466: Deleting WAL directory at /tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786289732823074-3431-0/minicluster-data/ts-1/wal/wals/f875addffcb84c74aab667cbd32c47b6
I20260809 15:38:21.467439 19306 log.cc:1196] T f875addffcb84c74aab667cbd32c47b6 P 421cb337efc747e292ba53a026c44c02: Deleting WAL directory at /tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786289732823074-3431-0/minicluster-data/ts-2/wal/wals/f875addffcb84c74aab667cbd32c47b6
I20260809 15:38:21.467701 19307 ts_tablet_manager.cc:1939] T f875addffcb84c74aab667cbd32c47b6 P 273a5fb47ef740c196b1a55c87ed7466: Deleting consensus metadata
I20260809 15:38:21.468261 19306 ts_tablet_manager.cc:1939] T f875addffcb84c74aab667cbd32c47b6 P 421cb337efc747e292ba53a026c44c02: Deleting consensus metadata
I20260809 15:38:21.468969 18761 catalog_manager.cc:4841] TS 273a5fb47ef740c196b1a55c87ed7466 (127.3.89.194:45639): tablet f875addffcb84c74aab667cbd32c47b6 (table test-table [id=f355e85709db40a29134bea82894c531]) successfully deleted
I20260809 15:38:21.469471 18759 catalog_manager.cc:4841] TS 421cb337efc747e292ba53a026c44c02 (127.3.89.195:40257): tablet f875addffcb84c74aab667cbd32c47b6 (table test-table [id=f355e85709db40a29134bea82894c531]) successfully deleted
W20260809 15:38:21.474838 18762 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.3.89.193:44707 (error 108)
I20260809 15:38:21.475147 3431 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskm314YR/build/debug/bin/kudu with pid 18984
I20260809 15:38:21.477731 19282 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:38:21.477423 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.89.193:44707 (local address 127.0.0.1:56898)
0809 15:38:21.477557 (+ 134us) negotiation.cc:107] Waiting for socket to connect
0809 15:38:21.477631 (+ 74us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.89.193:44707: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":53}
W20260809 15:38:21.477905 18762 catalog_manager.cc:4584] TS b0a56d93c098468296a2000c9c5609ca (127.3.89.193:44707): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet f875addffcb84c74aab667cbd32c47b6: Network error: Client connection negotiation failed: client connection to 127.3.89.193:44707: connect: Connection refused (error 111)
I20260809 15:38:21.482115 3431 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskm314YR/build/debug/bin/kudu with pid 19132
I20260809 15:38:21.488940 3431 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskm314YR/build/debug/bin/kudu with pid 18740
I20260809 15:38:21.498212 3431 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskm314YR/build/debug/bin/kudu
/tmp/dist-test-taskm314YR/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786289732823074-3431-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786289732823074-3431-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786289732823074-3431-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786289732823074-3431-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.3.89.254:33497
--webserver_interface=127.3.89.254
--webserver_port=38433
--keytab_file=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786289732823074-3431-0/krb5kdc/kudu_127.3.89.254.keytab
--principal=kudu/127.3.89.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.3.89.212:43771
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.3.89.254:33497
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786289732823074-3431-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786289732823074-3431-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786289732823074-3431-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 15:38:21.607589 19310 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 15:38:21.607832 19310 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 15:38:21.607872 19310 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 15:38:21.611775 19310 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 15:38:21.611824 19310 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 15:38:21.611842 19310 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 15:38:21.611860 19310 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 15:38:21.611874 19310 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 15:38:21.617027 19310 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.3.89.212:43771
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786289732823074-3431-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786289732823074-3431-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.3.89.254:33497
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786289732823074-3431-0/krb5kdc/kudu_127.3.89.254.keytab
--principal=kudu/127.3.89.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.3.89.254:33497
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786289732823074-3431-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.3.89.254
--webserver_port=38433
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.19310
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786289732823074-3431-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 09 Aug 2026 14:46:55 UTC on ccycloud.quasar-yswkmf.root.comops.site
build id 4576540
I20260809 15:38:21.617929 19310 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 15:38:21.618847 19310 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 15:38:21.627210 19316 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 15:38:21.627213 19319 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 15:38:21.627213 19317 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 15:38:21.627453 19310 server_base.cc:1034] running on GCE node
I20260809 15:38:21.628118 19310 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 15:38:21.629138 19310 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 15:38:21.630308 19310 hybrid_clock.cc:648] HybridClock initialized: now 1786289901630276 us; error 47 us; skew 500 ppm
Aug 09 15:38:21 dist-test-slave-2w3w krb5kdc[18158](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786289901, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.89.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 15:38:21.633158 19310 init.cc:377] Logged in from keytab as kudu/127.3.89.254@KRBTEST.COM (short username kudu)
I20260809 15:38:21.634037 19310 webserver.cc:460] Webserver started at http://127.3.89.254:38433/ using document root <none> and password file <none>
I20260809 15:38:21.634545 19310 fs_manager.cc:359] Metadata directory not provided
I20260809 15:38:21.634608 19310 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 15:38:21.638486 19310 fs_manager.cc:709] Time spent opening directory manager: real 0.002s user 0.002s sys 0.001s
I20260809 15:38:21.640578 19327 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 15:38:21.641805 19310 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260809 15:38:21.641945 19310 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786289732823074-3431-0/minicluster-data/master-0/data,/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786289732823074-3431-0/minicluster-data/master-0/wal
uuid: "f5ecfca57914460d8be340f7b52004b5"
format_stamp: "Formatted at 2026-08-09 15:38:19 on dist-test-slave-2w3w"
server_key: "aaf2376a0c2aedc840940ad38cd4d160"
server_key_iv: "e61ba99cabf8f23507a5d3c83dc759f0"
server_key_version: "encryptionkey@0"
I20260809 15:38:21.642513 19310 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786289732823074-3431-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786289732823074-3431-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786289732823074-3431-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 15:38:21.657706 19310 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 15:38:21.660995 19310 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 15:38:21.661198 19310 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 15:38:21.668452 19310 rpc_server.cc:307] RPC server started. Bound to: 127.3.89.254:33497
I20260809 15:38:21.668479 19383 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.3.89.254:33497 every 8 connection(s)
I20260809 15:38:21.669466 19310 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786289732823074-3431-0/minicluster-data/master-0/data/info.pb
I20260809 15:38:21.673568 19384 sys_catalog.cc:263] Verifying existing consensus state
I20260809 15:38:21.675256 3431 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskm314YR/build/debug/bin/kudu as pid 19310
I20260809 15:38:21.675369 19384 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f5ecfca57914460d8be340f7b52004b5: Bootstrap starting.
I20260809 15:38:21.675757 3431 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskm314YR/build/debug/bin/kudu
/tmp/dist-test-taskm314YR/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786289732823074-3431-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786289732823074-3431-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786289732823074-3431-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786289732823074-3431-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.3.89.193:44707
--local_ip_for_outbound_sockets=127.3.89.193
--tserver_master_addrs=127.3.89.254:33497
--webserver_port=42811
--webserver_interface=127.3.89.193
--keytab_file=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786289732823074-3431-0/krb5kdc/kudu_127.3.89.193.keytab
--principal=kudu/127.3.89.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.3.89.212:43771
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786289732823074-3431-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786289732823074-3431-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786289732823074-3431-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260809 15:38:21.694236 19384 log.cc:824] T 00000000000000000000000000000000 P f5ecfca57914460d8be340f7b52004b5: Log is configured to *not* fsync() on all Append() calls
I20260809 15:38:21.705487 19384 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f5ecfca57914460d8be340f7b52004b5: Bootstrap replayed 1/1 log segments. Stats: ops{read=14 overwritten=0 applied=14 ignored=0} inserts{seen=7 ignored=0} mutations{seen=9 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260809 15:38:21.705894 19384 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f5ecfca57914460d8be340f7b52004b5: Bootstrap complete.
I20260809 15:38:21.710282 19384 raft_consensus.cc:348] T 00000000000000000000000000000000 P f5ecfca57914460d8be340f7b52004b5 [term 2 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f5ecfca57914460d8be340f7b52004b5" member_type: VOTER last_known_addr { host: "127.3.89.254" port: 33497 } }
I20260809 15:38:21.710767 19384 raft_consensus.cc:729] T 00000000000000000000000000000000 P f5ecfca57914460d8be340f7b52004b5 [term 2 FOLLOWER]: Becoming Follower/Learner. State: Replica: f5ecfca57914460d8be340f7b52004b5, State: Initialized, Role: FOLLOWER
I20260809 15:38:21.711305 19384 consensus_queue.cc:260] T 00000000000000000000000000000000 P f5ecfca57914460d8be340f7b52004b5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 14, Last appended: 2.14, Last appended by leader: 14, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f5ecfca57914460d8be340f7b52004b5" member_type: VOTER last_known_addr { host: "127.3.89.254" port: 33497 } }
I20260809 15:38:21.711481 19384 raft_consensus.cc:388] T 00000000000000000000000000000000 P f5ecfca57914460d8be340f7b52004b5 [term 2 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 15:38:21.711560 19384 raft_consensus.cc:482] T 00000000000000000000000000000000 P f5ecfca57914460d8be340f7b52004b5 [term 2 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 15:38:21.711681 19384 raft_consensus.cc:3037] T 00000000000000000000000000000000 P f5ecfca57914460d8be340f7b52004b5 [term 2 FOLLOWER]: Advancing to term 3
I20260809 15:38:21.713637 19384 raft_consensus.cc:504] T 00000000000000000000000000000000 P f5ecfca57914460d8be340f7b52004b5 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f5ecfca57914460d8be340f7b52004b5" member_type: VOTER last_known_addr { host: "127.3.89.254" port: 33497 } }
I20260809 15:38:21.713977 19384 leader_election.cc:302] T 00000000000000000000000000000000 P f5ecfca57914460d8be340f7b52004b5 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: f5ecfca57914460d8be340f7b52004b5; no voters:
I20260809 15:38:21.714329 19384 leader_election.cc:288] T 00000000000000000000000000000000 P f5ecfca57914460d8be340f7b52004b5 [CANDIDATE]: Term 3 election: Requested vote from peers
I20260809 15:38:21.714416 19389 raft_consensus.cc:2781] T 00000000000000000000000000000000 P f5ecfca57914460d8be340f7b52004b5 [term 3 FOLLOWER]: Leader election won for term 3
I20260809 15:38:21.714607 19389 raft_consensus.cc:686] T 00000000000000000000000000000000 P f5ecfca57914460d8be340f7b52004b5 [term 3 LEADER]: Becoming Leader. State: Replica: f5ecfca57914460d8be340f7b52004b5, State: Running, Role: LEADER
I20260809 15:38:21.714918 19389 consensus_queue.cc:237] T 00000000000000000000000000000000 P f5ecfca57914460d8be340f7b52004b5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 2.14, Last appended by leader: 14, Current term: 3, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f5ecfca57914460d8be340f7b52004b5" member_type: VOTER last_known_addr { host: "127.3.89.254" port: 33497 } }
I20260809 15:38:21.715664 19384 sys_catalog.cc:564] T 00000000000000000000000000000000 P f5ecfca57914460d8be340f7b52004b5 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 15:38:21.716315 19390 sys_catalog.cc:455] T 00000000000000000000000000000000 P f5ecfca57914460d8be340f7b52004b5 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 3 leader_uuid: "f5ecfca57914460d8be340f7b52004b5" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f5ecfca57914460d8be340f7b52004b5" member_type: VOTER last_known_addr { host: "127.3.89.254" port: 33497 } } }
I20260809 15:38:21.716715 19390 sys_catalog.cc:458] T 00000000000000000000000000000000 P f5ecfca57914460d8be340f7b52004b5 [sys.catalog]: This master's current role is: LEADER
I20260809 15:38:21.716624 19391 sys_catalog.cc:455] T 00000000000000000000000000000000 P f5ecfca57914460d8be340f7b52004b5 [sys.catalog]: SysCatalogTable state changed. Reason: New leader f5ecfca57914460d8be340f7b52004b5. Latest consensus state: current_term: 3 leader_uuid: "f5ecfca57914460d8be340f7b52004b5" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f5ecfca57914460d8be340f7b52004b5" member_type: VOTER last_known_addr { host: "127.3.89.254" port: 33497 } } }
I20260809 15:38:21.717041 19391 sys_catalog.cc:458] T 00000000000000000000000000000000 P f5ecfca57914460d8be340f7b52004b5 [sys.catalog]: This master's current role is: LEADER
I20260809 15:38:21.717332 19399 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 15:38:21.722747 19399 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 15:38:21.723603 19399 catalog_manager.cc:1259] Loaded cluster ID: 6b5d21f311dd4f3e809e5ae566290b34
I20260809 15:38:21.723665 19399 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
F20260809 15:38:21.724584 19399 catalog_manager.cc:1466] Initializing Kudu internal certificate authority failed: Runtime error: unable to load data from memory: error:0D0680A8:asn1 encoding routines:ASN1_CHECK_TLEN:wrong tag:tasn_dec.c:1220 error:0D06C03A:asn1 encoding routines:ASN1_D2I_EX_PRIMITIVE:nested asn1 error:tasn_dec.c:788 error:0D08303A:asn1 encoding routines:ASN1_TEMPLATE_NOEXP_D2I:nested asn1 error:tasn_dec.c:720:Field=version, Type=RSA error:04093004:rsa routines:OLD_RSA_PRIV_DECODE:RSA lib:rsa_ameth.c:121 error:0D0680A8:asn1 encoding routines:ASN1_CHECK_TLEN:wrong tag:tasn_dec.c:1220 error:0D06C03A:asn1 encoding routines:ASN1_D2I_EX_PRIMITIVE:nested asn1 error:tasn_dec.c:788 error:0D08303A:asn1 encoding routines:ASN1_TEMPLATE_NOEXP_D2I:nested asn1 error:tasn_dec.c:720:Field=version, Type=PKCS8_PRIV_KEY_INFO
*** Check failure stack trace: ***
@ 0x7f6f6df209ad google::LogMessage::Fail() at ??:0
@ 0x7f6f6df2388a google::LogMessage::SendToLog() at ??:0
@ 0x7f6f6df204de google::LogMessage::Flush() at ??:0
@ 0x7f6f6df21a19 google::LogMessageFatal::~LogMessageFatal() at ??:0
@ 0x7f6f78863fdd _ZZN4kudu6master14CatalogManager24PrepareForLeadershipTaskEvENKUlSt8functionIFNS_6StatusEvEERKNS_9consensus13RaftConsensusElPKcE_clES5_S9_lSB_ at ??:0
@ 0x7f6f78864d2f kudu::master::CatalogManager::PrepareForLeadershipTask() at ??:0
@ 0x7f6f78861579 _ZZN4kudu6master14CatalogManager17ElectedAsLeaderCbEvENKUlvE_clEv at ??:0
@ 0x7f6f788a5640 _ZNSt17_Function_handlerIFvvEZN4kudu6master14CatalogManager17ElectedAsLeaderCbEvEUlvE_E9_M_invokeERKSt9_Any_data at ??:0
@ 0x61d838 std::function<>::operator()() at ??:0
@ 0x7f6f701723d7 kudu::ThreadPool::DispatchThread() at ??:0
@ 0x7f6f70172c83 _ZZN4kudu10ThreadPool12CreateThreadEvENKUlvE_clEv at ??:0
@ 0x7f6f701743f2 _ZNSt17_Function_handlerIFvvEZN4kudu10ThreadPool12CreateThreadEvEUlvE_E9_M_invokeERKSt9_Any_data at ??:0
@ 0x61d838 std::function<>::operator()() at ??:0
@ 0x7f6f70163512 kudu::Thread::SuperviseThread() at ??:0
@ 0x7f6f717a76db start_thread at ??:0
@ 0x7f6f6ca1e61f clone at ??:0
W20260809 15:38:21.810540 19387 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 15:38:21.810859 19387 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 15:38:21.810948 19387 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 15:38:21.815218 19387 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 15:38:21.815275 19387 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 15:38:21.815361 19387 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.3.89.193
I20260809 15:38:21.820606 19387 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.3.89.212:43771
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786289732823074-3431-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786289732823074-3431-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786289732823074-3431-0/krb5kdc/kudu_127.3.89.193.keytab
--principal=kudu/127.3.89.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.3.89.193:44707
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786289732823074-3431-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.3.89.193
--webserver_port=42811
--tserver_master_addrs=127.3.89.254:33497
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.19387
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.3.89.193
--log_dir=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786289732823074-3431-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 09 Aug 2026 14:46:55 UTC on ccycloud.quasar-yswkmf.root.comops.site
build id 4576540
I20260809 15:38:21.821756 19387 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 15:38:21.822892 19387 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 15:38:21.832677 19427 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 15:38:21.832635 19426 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 15:38:21.832892 19387 server_base.cc:1034] running on GCE node
W20260809 15:38:21.832646 19429 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 15:38:21.833478 19387 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 15:38:21.834262 19387 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 15:38:21.835484 19387 hybrid_clock.cc:648] HybridClock initialized: now 1786289901835448 us; error 44 us; skew 500 ppm
Aug 09 15:38:21 dist-test-slave-2w3w krb5kdc[18158](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786289901, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.89.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 15:38:21.838822 19387 init.cc:377] Logged in from keytab as kudu/127.3.89.193@KRBTEST.COM (short username kudu)
I20260809 15:38:21.839706 19387 webserver.cc:460] Webserver started at http://127.3.89.193:42811/ using document root <none> and password file <none>
I20260809 15:38:21.840243 19387 fs_manager.cc:359] Metadata directory not provided
I20260809 15:38:21.840304 19387 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 15:38:21.844130 19387 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.000s sys 0.003s
I20260809 15:38:21.846359 19437 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 15:38:21.847718 19387 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260809 15:38:21.847870 19387 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786289732823074-3431-0/minicluster-data/ts-0/data,/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786289732823074-3431-0/minicluster-data/ts-0/wal
uuid: "b0a56d93c098468296a2000c9c5609ca"
format_stamp: "Formatted at 2026-08-09 15:38:19 on dist-test-slave-2w3w"
server_key: "f0ee155f811f0aa13c8b3b8f3e49f2bb"
server_key_iv: "5e2a5a61d21e9b6c20da06f34bb7cbf2"
server_key_version: "encryptionkey@0"
I20260809 15:38:21.848459 19387 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786289732823074-3431-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786289732823074-3431-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786289732823074-3431-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 15:38:21.870801 19387 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 15:38:21.874617 19387 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 15:38:21.874866 19387 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 15:38:21.875515 19387 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 15:38:21.876677 19444 ts_tablet_manager.cc:536] Loading tablet metadata (0/1 complete)
I20260809 15:38:21.881958 19387 ts_tablet_manager.cc:579] Loaded tablet metadata (1 total tablets, 1 live tablets)
I20260809 15:38:21.882045 19387 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.006s user 0.000s sys 0.000s
I20260809 15:38:21.882129 19387 ts_tablet_manager.cc:594] Registering tablets (0/1 complete)
I20260809 15:38:21.883940 19387 ts_tablet_manager.cc:610] Registered 1 tablets
I20260809 15:38:21.884016 19387 ts_tablet_manager.cc:589] Time spent register tablets: real 0.002s user 0.001s sys 0.000s
I20260809 15:38:21.884374 19444 tablet_bootstrap.cc:492] T f875addffcb84c74aab667cbd32c47b6 P b0a56d93c098468296a2000c9c5609ca: Bootstrap starting.
I20260809 15:38:21.894713 19387 rpc_server.cc:307] RPC server started. Bound to: 127.3.89.193:44707
I20260809 15:38:21.894948 19560 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.3.89.193:44707 every 8 connection(s)
I20260809 15:38:21.895740 19387 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786289732823074-3431-0/minicluster-data/ts-0/data/info.pb
I20260809 15:38:21.903807 3431 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskm314YR/build/debug/bin/kudu as pid 19387
I20260809 15:38:21.904229 3431 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskm314YR/build/debug/bin/kudu
/tmp/dist-test-taskm314YR/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786289732823074-3431-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786289732823074-3431-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786289732823074-3431-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786289732823074-3431-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.3.89.194:45639
--local_ip_for_outbound_sockets=127.3.89.194
--tserver_master_addrs=127.3.89.254:33497
--webserver_port=40495
--webserver_interface=127.3.89.194
--keytab_file=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786289732823074-3431-0/krb5kdc/kudu_127.3.89.194.keytab
--principal=kudu/127.3.89.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.3.89.212:43771
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786289732823074-3431-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786289732823074-3431-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786289732823074-3431-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260809 15:38:21.906385 19444 log.cc:824] T f875addffcb84c74aab667cbd32c47b6 P b0a56d93c098468296a2000c9c5609ca: Log is configured to *not* fsync() on all Append() calls
I20260809 15:38:21.913242 19444 tablet_bootstrap.cc:492] T f875addffcb84c74aab667cbd32c47b6 P b0a56d93c098468296a2000c9c5609ca: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=1 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260809 15:38:21.913652 19444 tablet_bootstrap.cc:492] T f875addffcb84c74aab667cbd32c47b6 P b0a56d93c098468296a2000c9c5609ca: Bootstrap complete.
I20260809 15:38:21.914767 19444 ts_tablet_manager.cc:1397] T f875addffcb84c74aab667cbd32c47b6 P b0a56d93c098468296a2000c9c5609ca: Time spent bootstrapping tablet: real 0.031s user 0.022s sys 0.005s
I20260809 15:38:21.917102 19444 raft_consensus.cc:348] T f875addffcb84c74aab667cbd32c47b6 P b0a56d93c098468296a2000c9c5609ca [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "421cb337efc747e292ba53a026c44c02" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 40257 } } peers { permanent_uuid: "273a5fb47ef740c196b1a55c87ed7466" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 45639 } } peers { permanent_uuid: "b0a56d93c098468296a2000c9c5609ca" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 44707 } }
I20260809 15:38:21.917533 19444 raft_consensus.cc:729] T f875addffcb84c74aab667cbd32c47b6 P b0a56d93c098468296a2000c9c5609ca [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: b0a56d93c098468296a2000c9c5609ca, State: Initialized, Role: FOLLOWER
I20260809 15:38:21.918017 19444 consensus_queue.cc:260] T f875addffcb84c74aab667cbd32c47b6 P b0a56d93c098468296a2000c9c5609ca [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "421cb337efc747e292ba53a026c44c02" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 40257 } } peers { permanent_uuid: "273a5fb47ef740c196b1a55c87ed7466" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 45639 } } peers { permanent_uuid: "b0a56d93c098468296a2000c9c5609ca" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 44707 } }
I20260809 15:38:21.919003 19444 ts_tablet_manager.cc:1428] T f875addffcb84c74aab667cbd32c47b6 P b0a56d93c098468296a2000c9c5609ca: Time spent starting tablet: real 0.004s user 0.007s sys 0.000s
W20260809 15:38:21.970292 19563 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 15:38:21.897649 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.89.254:33497 (local address 127.3.89.193:47301)
0809 15:38:21.898122 (+ 473us) negotiation.cc:107] Waiting for socket to connect
0809 15:38:21.898177 (+ 55us) client_negotiation.cc:174] Beginning negotiation
0809 15:38:21.899020 (+ 843us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 15:38:21.969470 (+ 70450us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.89.254:33497: BlockingRecv error: recv error from unknown peer: Transport endpoint is not connected (error 107)
Metrics: {"client-negotiator.queue_time_us":276,"thread_start_us":151,"threads_started":1}
W20260809 15:38:21.970909 19561 heartbeater.cc:644] Failed to heartbeat to 127.3.89.254:33497 (0 consecutive failures): Network error: Failed to ping master at 127.3.89.254:33497: Client connection negotiation failed: client connection to 127.3.89.254:33497: BlockingRecv error: recv error from unknown peer: Transport endpoint is not connected (error 107)
I20260809 15:38:21.971668 19563 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:38:21.971405 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.89.254:33497 (local address 127.3.89.193:58581)
0809 15:38:21.971539 (+ 134us) negotiation.cc:107] Waiting for socket to connect
0809 15:38:21.971615 (+ 76us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.89.254:33497: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":55}
I20260809 15:38:21.972445 19563 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:38:21.972192 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.89.254:33497 (local address 127.3.89.193:35559)
0809 15:38:21.972324 (+ 132us) negotiation.cc:107] Waiting for socket to connect
0809 15:38:21.972384 (+ 60us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.89.254:33497: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":51}
W20260809 15:38:21.972635 19561 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260809 15:38:22.035748 19564 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 15:38:22.036037 19564 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 15:38:22.036118 19564 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 15:38:22.040253 19564 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 15:38:22.040339 19564 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 15:38:22.040436 19564 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.3.89.194
I20260809 15:38:22.045667 19564 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.3.89.212:43771
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786289732823074-3431-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786289732823074-3431-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786289732823074-3431-0/krb5kdc/kudu_127.3.89.194.keytab
--principal=kudu/127.3.89.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.3.89.194:45639
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786289732823074-3431-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.3.89.194
--webserver_port=40495
--tserver_master_addrs=127.3.89.254:33497
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.19564
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.3.89.194
--log_dir=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786289732823074-3431-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 09 Aug 2026 14:46:55 UTC on ccycloud.quasar-yswkmf.root.comops.site
build id 4576540
I20260809 15:38:22.046660 19564 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 15:38:22.047770 19564 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 15:38:22.057408 19571 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 15:38:22.057420 19574 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 15:38:22.057475 19564 server_base.cc:1034] running on GCE node
W20260809 15:38:22.057425 19572 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 15:38:22.058053 19564 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 15:38:22.058740 19564 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 15:38:22.059922 19564 hybrid_clock.cc:648] HybridClock initialized: now 1786289902059893 us; error 37 us; skew 500 ppm
Aug 09 15:38:22 dist-test-slave-2w3w krb5kdc[18158](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786289902, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.89.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 15:38:22.062572 19564 init.cc:377] Logged in from keytab as kudu/127.3.89.194@KRBTEST.COM (short username kudu)
I20260809 15:38:22.063431 19564 webserver.cc:460] Webserver started at http://127.3.89.194:40495/ using document root <none> and password file <none>
I20260809 15:38:22.063948 19564 fs_manager.cc:359] Metadata directory not provided
I20260809 15:38:22.064007 19564 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 15:38:22.067545 19564 fs_manager.cc:709] Time spent opening directory manager: real 0.002s user 0.003s sys 0.000s
I20260809 15:38:22.069473 19582 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 15:38:22.070652 19564 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260809 15:38:22.070775 19564 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786289732823074-3431-0/minicluster-data/ts-1/data,/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786289732823074-3431-0/minicluster-data/ts-1/wal
uuid: "273a5fb47ef740c196b1a55c87ed7466"
format_stamp: "Formatted at 2026-08-09 15:38:19 on dist-test-slave-2w3w"
server_key: "c6c967900852bcc2f6e79e6f877d02d2"
server_key_iv: "9d091753c3ded8f1cc2acd366d1e459d"
server_key_version: "encryptionkey@0"
I20260809 15:38:22.071439 19564 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786289732823074-3431-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786289732823074-3431-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786289732823074-3431-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 15:38:22.084720 19564 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 15:38:22.087826 19564 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 15:38:22.088047 19564 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 15:38:22.088646 19564 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 15:38:22.089494 19564 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 15:38:22.089541 19564 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 15:38:22.089608 19564 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 15:38:22.089643 19564 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 15:38:22.098865 19564 rpc_server.cc:307] RPC server started. Bound to: 127.3.89.194:45639
I20260809 15:38:22.098889 19708 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.3.89.194:45639 every 8 connection(s)
I20260809 15:38:22.099858 19564 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786289732823074-3431-0/minicluster-data/ts-1/data/info.pb
I20260809 15:38:22.102581 3431 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskm314YR/build/debug/bin/kudu as pid 19564
I20260809 15:38:22.103166 3431 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskm314YR/build/debug/bin/kudu
/tmp/dist-test-taskm314YR/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786289732823074-3431-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786289732823074-3431-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786289732823074-3431-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786289732823074-3431-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.3.89.195:40257
--local_ip_for_outbound_sockets=127.3.89.195
--tserver_master_addrs=127.3.89.254:33497
--webserver_port=32809
--webserver_interface=127.3.89.195
--keytab_file=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786289732823074-3431-0/krb5kdc/kudu_127.3.89.195.keytab
--principal=kudu/127.3.89.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.3.89.212:43771
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786289732823074-3431-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786289732823074-3431-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786289732823074-3431-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260809 15:38:22.104120 19711 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:38:22.101943 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.89.254:33497 (local address 127.3.89.194:60431)
0809 15:38:22.102456 (+ 513us) negotiation.cc:107] Waiting for socket to connect
0809 15:38:22.103176 (+ 720us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.89.254:33497: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":267,"thread_start_us":112,"threads_started":1}
W20260809 15:38:22.104578 19709 heartbeater.cc:644] Failed to heartbeat to 127.3.89.254:33497 (0 consecutive failures): Network error: Failed to ping master at 127.3.89.254:33497: Client connection negotiation failed: client connection to 127.3.89.254:33497: connect: Connection refused (error 111)
I20260809 15:38:22.105270 19711 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:38:22.105016 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.89.254:33497 (local address 127.3.89.194:35311)
0809 15:38:22.105164 (+ 148us) negotiation.cc:107] Waiting for socket to connect
0809 15:38:22.105215 (+ 51us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.89.254:33497: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":57}
I20260809 15:38:22.105984 19711 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:38:22.105791 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.89.254:33497 (local address 127.3.89.194:41663)
0809 15:38:22.105891 (+ 100us) negotiation.cc:107] Waiting for socket to connect
0809 15:38:22.105933 (+ 42us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.89.254:33497: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":44}
W20260809 15:38:22.106233 19709 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260809 15:38:22.215936 19713 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 15:38:22.216180 19713 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 15:38:22.216223 19713 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 15:38:22.220034 19713 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 15:38:22.220080 19713 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 15:38:22.220136 19713 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.3.89.195
I20260809 15:38:22.225248 19713 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.3.89.212:43771
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786289732823074-3431-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786289732823074-3431-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786289732823074-3431-0/krb5kdc/kudu_127.3.89.195.keytab
--principal=kudu/127.3.89.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.3.89.195:40257
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786289732823074-3431-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.3.89.195
--webserver_port=32809
--tserver_master_addrs=127.3.89.254:33497
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.19713
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.3.89.195
--log_dir=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786289732823074-3431-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 09 Aug 2026 14:46:55 UTC on ccycloud.quasar-yswkmf.root.comops.site
build id 4576540
I20260809 15:38:22.226251 19713 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 15:38:22.227192 19713 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 15:38:22.236475 19718 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 15:38:22.236491 19721 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 15:38:22.236483 19719 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 15:38:22.236814 19713 server_base.cc:1034] running on GCE node
I20260809 15:38:22.237315 19713 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 15:38:22.237936 19713 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 15:38:22.239095 19713 hybrid_clock.cc:648] HybridClock initialized: now 1786289902239084 us; error 23 us; skew 500 ppm
Aug 09 15:38:22 dist-test-slave-2w3w krb5kdc[18158](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786289902, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.89.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 15:38:22.241725 19713 init.cc:377] Logged in from keytab as kudu/127.3.89.195@KRBTEST.COM (short username kudu)
I20260809 15:38:22.242480 19713 webserver.cc:460] Webserver started at http://127.3.89.195:32809/ using document root <none> and password file <none>
I20260809 15:38:22.243023 19713 fs_manager.cc:359] Metadata directory not provided
I20260809 15:38:22.243098 19713 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 15:38:22.246636 19713 fs_manager.cc:709] Time spent opening directory manager: real 0.002s user 0.003s sys 0.000s
I20260809 15:38:22.248591 19729 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 15:38:22.250308 19713 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260809 15:38:22.250427 19713 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786289732823074-3431-0/minicluster-data/ts-2/data,/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786289732823074-3431-0/minicluster-data/ts-2/wal
uuid: "421cb337efc747e292ba53a026c44c02"
format_stamp: "Formatted at 2026-08-09 15:38:19 on dist-test-slave-2w3w"
server_key: "eb2f06f9d711076fc80f282898368483"
server_key_iv: "b0314ff48b906d23f3b0478ecc39a789"
server_key_version: "encryptionkey@0"
I20260809 15:38:22.251153 19713 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786289732823074-3431-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786289732823074-3431-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786289732823074-3431-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 15:38:22.273882 19713 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 15:38:22.276922 19713 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 15:38:22.277077 19713 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 15:38:22.277595 19713 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 15:38:22.278362 19713 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 15:38:22.278410 19713 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 15:38:22.278450 19713 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 15:38:22.278465 19713 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 15:38:22.287694 19713 rpc_server.cc:307] RPC server started. Bound to: 127.3.89.195:40257
I20260809 15:38:22.287729 19855 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.3.89.195:40257 every 8 connection(s)
I20260809 15:38:22.288669 19713 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786289732823074-3431-0/minicluster-data/ts-2/data/info.pb
I20260809 15:38:22.290661 3431 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskm314YR/build/debug/bin/kudu as pid 19713
I20260809 15:38:22.292775 19858 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:38:22.290859 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.89.254:33497 (local address 127.3.89.195:43249)
0809 15:38:22.291490 (+ 631us) negotiation.cc:107] Waiting for socket to connect
0809 15:38:22.292090 (+ 600us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.89.254:33497: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":381,"thread_start_us":156,"threads_started":1}
W20260809 15:38:22.293220 19856 heartbeater.cc:644] Failed to heartbeat to 127.3.89.254:33497 (0 consecutive failures): Network error: Failed to ping master at 127.3.89.254:33497: Client connection negotiation failed: client connection to 127.3.89.254:33497: connect: Connection refused (error 111)
I20260809 15:38:22.293809 19858 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:38:22.293600 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.89.254:33497 (local address 127.3.89.195:45555)
0809 15:38:22.293705 (+ 105us) negotiation.cc:107] Waiting for socket to connect
0809 15:38:22.293752 (+ 47us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.89.254:33497: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":42}
I20260809 15:38:22.294389 19858 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:38:22.294196 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.89.254:33497 (local address 127.3.89.195:58499)
0809 15:38:22.294301 (+ 105us) negotiation.cc:107] Waiting for socket to connect
0809 15:38:22.294341 (+ 40us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.89.254:33497: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":42}
W20260809 15:38:22.294576 19856 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
I20260809 15:38:22.973625 19860 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:38:22.973194 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.89.254:33497 (local address 127.3.89.193:48707)
0809 15:38:22.973458 (+ 264us) negotiation.cc:107] Waiting for socket to connect
0809 15:38:22.973515 (+ 57us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.89.254:33497: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":191,"thread_start_us":75,"threads_started":1}
I20260809 15:38:23.066641 19861 raft_consensus.cc:482] T f875addffcb84c74aab667cbd32c47b6 P b0a56d93c098468296a2000c9c5609ca [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 15:38:23.066880 19861 raft_consensus.cc:504] T f875addffcb84c74aab667cbd32c47b6 P b0a56d93c098468296a2000c9c5609ca [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "421cb337efc747e292ba53a026c44c02" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 40257 } } peers { permanent_uuid: "273a5fb47ef740c196b1a55c87ed7466" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 45639 } } peers { permanent_uuid: "b0a56d93c098468296a2000c9c5609ca" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 44707 } }
I20260809 15:38:23.068154 19861 leader_election.cc:288] T f875addffcb84c74aab667cbd32c47b6 P b0a56d93c098468296a2000c9c5609ca [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 421cb337efc747e292ba53a026c44c02 (127.3.89.195:40257), 273a5fb47ef740c196b1a55c87ed7466 (127.3.89.194:45639)
Aug 09 15:38:23 dist-test-slave-2w3w krb5kdc[18158](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786289901, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.89.193@KRBTEST.COM for kudu/127.3.89.195@KRBTEST.COM
Aug 09 15:38:23 dist-test-slave-2w3w krb5kdc[18158](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786289901, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.89.193@KRBTEST.COM for kudu/127.3.89.194@KRBTEST.COM
I20260809 15:38:23.082870 19862 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:38:23.068374 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.89.194:45639 (local address 127.3.89.193:47671)
0809 15:38:23.068715 (+ 341us) negotiation.cc:107] Waiting for socket to connect
0809 15:38:23.068729 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0809 15:38:23.068842 (+ 113us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 15:38:23.070360 (+ 1518us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 15:38:23.070366 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 15:38:23.070899 (+ 533us) client_negotiation.cc:189] Negotiated authn=SASL
0809 15:38:23.071421 (+ 522us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:38:23.071434 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:38:23.072741 (+ 1307us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:38:23.072744 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:38:23.073636 (+ 892us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:38:23.073645 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:38:23.074347 (+ 702us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:38:23.074350 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:38:23.074585 (+ 235us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:38:23.076028 (+ 1443us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 15:38:23.076043 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0809 15:38:23.077500 (+ 1457us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 15:38:23.080451 (+ 2951us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 15:38:23.080457 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 15:38:23.080461 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0809 15:38:23.080859 (+ 398us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 15:38:23.081247 (+ 388us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 15:38:23.081250 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 15:38:23.081252 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 15:38:23.081327 (+ 75us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 15:38:23.081921 (+ 594us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 15:38:23.081926 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 15:38:23.082164 (+ 238us) client_negotiation.cc:769] Sending connection context
0809 15:38:23.082347 (+ 183us) client_negotiation.cc:240] Negotiation successful
0809 15:38:23.082670 (+ 323us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":279,"mutex_wait_us":140,"spinlock_wait_cycles":138880,"thread_start_us":77,"threads_started":1}
I20260809 15:38:23.082885 19863 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:38:23.068501 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.89.193:53645 (local address 127.3.89.195:40257)
0809 15:38:23.068941 (+ 440us) server_negotiation.cc:206] Beginning negotiation
0809 15:38:23.068958 (+ 17us) server_negotiation.cc:399] Waiting for connection header
0809 15:38:23.069011 (+ 53us) server_negotiation.cc:407] Connection header received
0809 15:38:23.069646 (+ 635us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 15:38:23.069651 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 15:38:23.069997 (+ 346us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 15:38:23.070290 (+ 293us) server_negotiation.cc:226] Negotiated authn=SASL
0809 15:38:23.071619 (+ 1329us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:38:23.072571 (+ 952us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:38:23.073765 (+ 1194us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:38:23.074261 (+ 496us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:38:23.074445 (+ 184us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:38:23.077691 (+ 3246us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 15:38:23.077697 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 15:38:23.077712 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 15:38:23.077745 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0809 15:38:23.080258 (+ 2513us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 15:38:23.080866 (+ 608us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 15:38:23.080872 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 15:38:23.080879 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0809 15:38:23.081026 (+ 147us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 15:38:23.081406 (+ 380us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 15:38:23.081412 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 15:38:23.081413 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 15:38:23.081796 (+ 383us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 15:38:23.081941 (+ 145us) server_negotiation.cc:1035] Waiting for connection context
0809 15:38:23.082458 (+ 517us) server_negotiation.cc:299] Negotiation successful
0809 15:38:23.082707 (+ 249us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":315,"thread_start_us":112,"threads_started":1}
I20260809 15:38:23.082924 19860 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:38:23.068353 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.89.195:40257 (local address 127.3.89.193:53645)
0809 15:38:23.068501 (+ 148us) negotiation.cc:107] Waiting for socket to connect
0809 15:38:23.068518 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0809 15:38:23.068635 (+ 117us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 15:38:23.070635 (+ 2000us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 15:38:23.070637 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 15:38:23.070882 (+ 245us) client_negotiation.cc:189] Negotiated authn=SASL
0809 15:38:23.071420 (+ 538us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:38:23.071434 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:38:23.072730 (+ 1296us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:38:23.072734 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:38:23.073638 (+ 904us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:38:23.073648 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:38:23.074398 (+ 750us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:38:23.074402 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:38:23.074561 (+ 159us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:38:23.075405 (+ 844us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 15:38:23.075421 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0809 15:38:23.077177 (+ 1756us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 15:38:23.080399 (+ 3222us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 15:38:23.080407 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 15:38:23.080421 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0809 15:38:23.080749 (+ 328us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 15:38:23.081128 (+ 379us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 15:38:23.081132 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 15:38:23.081134 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 15:38:23.081301 (+ 167us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 15:38:23.081920 (+ 619us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 15:38:23.081926 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 15:38:23.082165 (+ 239us) client_negotiation.cc:769] Sending connection context
0809 15:38:23.082347 (+ 182us) client_negotiation.cc:240] Negotiation successful
0809 15:38:23.082756 (+ 409us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":59,"mutex_wait_us":263,"spinlock_wait_cycles":23552}
I20260809 15:38:23.083559 19864 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:38:23.068719 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.89.193:47671 (local address 127.3.89.194:45639)
0809 15:38:23.069061 (+ 342us) server_negotiation.cc:206] Beginning negotiation
0809 15:38:23.069071 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0809 15:38:23.069101 (+ 30us) server_negotiation.cc:407] Connection header received
0809 15:38:23.069694 (+ 593us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 15:38:23.069699 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 15:38:23.069997 (+ 298us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 15:38:23.070263 (+ 266us) server_negotiation.cc:226] Negotiated authn=SASL
0809 15:38:23.071605 (+ 1342us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:38:23.072576 (+ 971us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:38:23.073762 (+ 1186us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:38:23.074213 (+ 451us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:38:23.074399 (+ 186us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:38:23.077691 (+ 3292us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 15:38:23.077697 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 15:38:23.077712 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 15:38:23.077745 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0809 15:38:23.080258 (+ 2513us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 15:38:23.080987 (+ 729us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 15:38:23.080991 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 15:38:23.080995 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 15:38:23.081144 (+ 149us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 15:38:23.081448 (+ 304us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 15:38:23.081450 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 15:38:23.081452 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 15:38:23.081796 (+ 344us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 15:38:23.081942 (+ 146us) server_negotiation.cc:1035] Waiting for connection context
0809 15:38:23.082458 (+ 516us) server_negotiation.cc:299] Negotiation successful
0809 15:38:23.082745 (+ 287us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":241,"thread_start_us":89,"threads_started":1}
W20260809 15:38:23.085508 19438 leader_election.cc:341] T f875addffcb84c74aab667cbd32c47b6 P b0a56d93c098468296a2000c9c5609ca [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 421cb337efc747e292ba53a026c44c02 (127.3.89.195:40257): Not found: Tablet not found: f875addffcb84c74aab667cbd32c47b6
W20260809 15:38:23.085727 19440 leader_election.cc:341] T f875addffcb84c74aab667cbd32c47b6 P b0a56d93c098468296a2000c9c5609ca [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 273a5fb47ef740c196b1a55c87ed7466 (127.3.89.194:45639): Not found: Tablet not found: f875addffcb84c74aab667cbd32c47b6
I20260809 15:38:23.085816 19440 leader_election.cc:302] T f875addffcb84c74aab667cbd32c47b6 P b0a56d93c098468296a2000c9c5609ca [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b0a56d93c098468296a2000c9c5609ca; no voters: 273a5fb47ef740c196b1a55c87ed7466, 421cb337efc747e292ba53a026c44c02
I20260809 15:38:23.086068 19861 raft_consensus.cc:2726] T f875addffcb84c74aab667cbd32c47b6 P b0a56d93c098468296a2000c9c5609ca [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260809 15:38:23.107241 19865 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:38:23.106822 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.89.254:33497 (local address 127.3.89.194:51237)
0809 15:38:23.107093 (+ 271us) negotiation.cc:107] Waiting for socket to connect
0809 15:38:23.107151 (+ 58us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.89.254:33497: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":205,"thread_start_us":71,"threads_started":1}
I20260809 15:38:23.295575 19866 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:38:23.295140 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.89.254:33497 (local address 127.3.89.195:36811)
0809 15:38:23.295408 (+ 268us) negotiation.cc:107] Waiting for socket to connect
0809 15:38:23.295476 (+ 68us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.89.254:33497: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":196,"thread_start_us":92,"threads_started":1}
I20260809 15:38:23.974817 19868 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:38:23.974372 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.89.254:33497 (local address 127.3.89.193:58705)
0809 15:38:23.974658 (+ 286us) negotiation.cc:107] Waiting for socket to connect
0809 15:38:23.974710 (+ 52us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.89.254:33497: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":215,"thread_start_us":108,"threads_started":1}
I20260809 15:38:24.108386 19869 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:38:24.107974 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.89.254:33497 (local address 127.3.89.194:36601)
0809 15:38:24.108225 (+ 251us) negotiation.cc:107] Waiting for socket to connect
0809 15:38:24.108281 (+ 56us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.89.254:33497: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":182,"thread_start_us":90,"threads_started":1}
I20260809 15:38:24.296764 19870 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:38:24.296314 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.89.254:33497 (local address 127.3.89.195:40033)
0809 15:38:24.296564 (+ 250us) negotiation.cc:107] Waiting for socket to connect
0809 15:38:24.296638 (+ 74us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.89.254:33497: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":169,"thread_start_us":95,"threads_started":1}
I20260809 15:38:24.824484 19871 raft_consensus.cc:482] T f875addffcb84c74aab667cbd32c47b6 P b0a56d93c098468296a2000c9c5609ca [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 15:38:24.824610 19871 raft_consensus.cc:504] T f875addffcb84c74aab667cbd32c47b6 P b0a56d93c098468296a2000c9c5609ca [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "421cb337efc747e292ba53a026c44c02" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 40257 } } peers { permanent_uuid: "273a5fb47ef740c196b1a55c87ed7466" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 45639 } } peers { permanent_uuid: "b0a56d93c098468296a2000c9c5609ca" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 44707 } }
I20260809 15:38:24.825069 19871 leader_election.cc:288] T f875addffcb84c74aab667cbd32c47b6 P b0a56d93c098468296a2000c9c5609ca [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 421cb337efc747e292ba53a026c44c02 (127.3.89.195:40257), 273a5fb47ef740c196b1a55c87ed7466 (127.3.89.194:45639)
W20260809 15:38:24.825966 19438 leader_election.cc:341] T f875addffcb84c74aab667cbd32c47b6 P b0a56d93c098468296a2000c9c5609ca [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 421cb337efc747e292ba53a026c44c02 (127.3.89.195:40257): Not found: Tablet not found: f875addffcb84c74aab667cbd32c47b6
W20260809 15:38:24.826078 19440 leader_election.cc:341] T f875addffcb84c74aab667cbd32c47b6 P b0a56d93c098468296a2000c9c5609ca [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 273a5fb47ef740c196b1a55c87ed7466 (127.3.89.194:45639): Not found: Tablet not found: f875addffcb84c74aab667cbd32c47b6
I20260809 15:38:24.826143 19440 leader_election.cc:302] T f875addffcb84c74aab667cbd32c47b6 P b0a56d93c098468296a2000c9c5609ca [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b0a56d93c098468296a2000c9c5609ca; no voters: 273a5fb47ef740c196b1a55c87ed7466, 421cb337efc747e292ba53a026c44c02
I20260809 15:38:24.826270 19871 raft_consensus.cc:2726] T f875addffcb84c74aab667cbd32c47b6 P b0a56d93c098468296a2000c9c5609ca [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260809 15:38:24.975971 19872 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:38:24.975570 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.89.254:33497 (local address 127.3.89.193:50325)
0809 15:38:24.975809 (+ 239us) negotiation.cc:107] Waiting for socket to connect
0809 15:38:24.975864 (+ 55us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.89.254:33497: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":173,"thread_start_us":78,"threads_started":1}
I20260809 15:38:25.109542 19873 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:38:25.109113 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.89.254:33497 (local address 127.3.89.194:52713)
0809 15:38:25.109391 (+ 278us) negotiation.cc:107] Waiting for socket to connect
0809 15:38:25.109451 (+ 60us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.89.254:33497: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":199,"thread_start_us":75,"threads_started":1}
I20260809 15:38:25.297811 19874 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:38:25.297403 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.89.254:33497 (local address 127.3.89.195:56713)
0809 15:38:25.297664 (+ 261us) negotiation.cc:107] Waiting for socket to connect
0809 15:38:25.297718 (+ 54us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.89.254:33497: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":190,"thread_start_us":75,"threads_started":1}
I20260809 15:38:25.977095 19875 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:38:25.976684 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.89.254:33497 (local address 127.3.89.193:39665)
0809 15:38:25.976934 (+ 250us) negotiation.cc:107] Waiting for socket to connect
0809 15:38:25.976999 (+ 65us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.89.254:33497: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":182,"thread_start_us":84,"threads_started":1}
I20260809 15:38:26.110632 19876 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:38:26.110250 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.89.254:33497 (local address 127.3.89.194:52955)
0809 15:38:26.110497 (+ 247us) negotiation.cc:107] Waiting for socket to connect
0809 15:38:26.110548 (+ 51us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.89.254:33497: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":179,"thread_start_us":73,"threads_started":1}
I20260809 15:38:26.298929 19877 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:38:26.298532 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.89.254:33497 (local address 127.3.89.195:49121)
0809 15:38:26.298794 (+ 262us) negotiation.cc:107] Waiting for socket to connect
0809 15:38:26.298846 (+ 52us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.89.254:33497: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":196,"thread_start_us":85,"threads_started":1}
I20260809 15:38:26.835011 19878 raft_consensus.cc:482] T f875addffcb84c74aab667cbd32c47b6 P b0a56d93c098468296a2000c9c5609ca [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 15:38:26.835156 19878 raft_consensus.cc:504] T f875addffcb84c74aab667cbd32c47b6 P b0a56d93c098468296a2000c9c5609ca [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "421cb337efc747e292ba53a026c44c02" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 40257 } } peers { permanent_uuid: "273a5fb47ef740c196b1a55c87ed7466" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 45639 } } peers { permanent_uuid: "b0a56d93c098468296a2000c9c5609ca" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 44707 } }
I20260809 15:38:26.835639 19878 leader_election.cc:288] T f875addffcb84c74aab667cbd32c47b6 P b0a56d93c098468296a2000c9c5609ca [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 421cb337efc747e292ba53a026c44c02 (127.3.89.195:40257), 273a5fb47ef740c196b1a55c87ed7466 (127.3.89.194:45639)
W20260809 15:38:26.836539 19438 leader_election.cc:341] T f875addffcb84c74aab667cbd32c47b6 P b0a56d93c098468296a2000c9c5609ca [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 421cb337efc747e292ba53a026c44c02 (127.3.89.195:40257): Not found: Tablet not found: f875addffcb84c74aab667cbd32c47b6
W20260809 15:38:26.836652 19440 leader_election.cc:341] T f875addffcb84c74aab667cbd32c47b6 P b0a56d93c098468296a2000c9c5609ca [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 273a5fb47ef740c196b1a55c87ed7466 (127.3.89.194:45639): Not found: Tablet not found: f875addffcb84c74aab667cbd32c47b6
I20260809 15:38:26.836776 19440 leader_election.cc:302] T f875addffcb84c74aab667cbd32c47b6 P b0a56d93c098468296a2000c9c5609ca [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b0a56d93c098468296a2000c9c5609ca; no voters: 273a5fb47ef740c196b1a55c87ed7466, 421cb337efc747e292ba53a026c44c02
I20260809 15:38:26.836973 19878 raft_consensus.cc:2726] T f875addffcb84c74aab667cbd32c47b6 P b0a56d93c098468296a2000c9c5609ca [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260809 15:38:26.978507 19879 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:38:26.978027 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.89.254:33497 (local address 127.3.89.193:38223)
0809 15:38:26.978362 (+ 335us) negotiation.cc:107] Waiting for socket to connect
0809 15:38:26.978414 (+ 52us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.89.254:33497: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":269,"thread_start_us":134,"threads_started":1}
I20260809 15:38:27.111809 19880 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:38:27.111398 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.89.254:33497 (local address 127.3.89.194:55575)
0809 15:38:27.111654 (+ 256us) negotiation.cc:107] Waiting for socket to connect
0809 15:38:27.111710 (+ 56us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.89.254:33497: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":187,"thread_start_us":89,"threads_started":1}
I20260809 15:38:27.300086 19881 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:38:27.299693 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.89.254:33497 (local address 127.3.89.195:54219)
0809 15:38:27.299951 (+ 258us) negotiation.cc:107] Waiting for socket to connect
0809 15:38:27.300000 (+ 49us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.89.254:33497: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":181,"thread_start_us":108,"threads_started":1}
I20260809 15:38:27.979691 19882 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:38:27.979269 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.89.254:33497 (local address 127.3.89.193:44779)
0809 15:38:27.979538 (+ 269us) negotiation.cc:107] Waiting for socket to connect
0809 15:38:27.979597 (+ 59us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.89.254:33497: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":196,"thread_start_us":86,"threads_started":1}
I20260809 15:38:28.113070 19883 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:38:28.112610 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.89.254:33497 (local address 127.3.89.194:33175)
0809 15:38:28.112899 (+ 289us) negotiation.cc:107] Waiting for socket to connect
0809 15:38:28.112957 (+ 58us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.89.254:33497: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":213,"thread_start_us":80,"threads_started":1}
I20260809 15:38:28.301160 19884 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:38:28.300774 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.89.254:33497 (local address 127.3.89.195:41405)
0809 15:38:28.301018 (+ 244us) negotiation.cc:107] Waiting for socket to connect
0809 15:38:28.301070 (+ 52us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.89.254:33497: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":170,"thread_start_us":72,"threads_started":1}
I20260809 15:38:28.980988 19885 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:38:28.980534 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.89.254:33497 (local address 127.3.89.193:51895)
0809 15:38:28.980843 (+ 309us) negotiation.cc:107] Waiting for socket to connect
0809 15:38:28.980896 (+ 53us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.89.254:33497: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":219,"thread_start_us":94,"threads_started":1}
I20260809 15:38:29.114241 19886 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:38:29.113819 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.89.254:33497 (local address 127.3.89.194:60435)
0809 15:38:29.114095 (+ 276us) negotiation.cc:107] Waiting for socket to connect
0809 15:38:29.114147 (+ 52us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.89.254:33497: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":209,"thread_start_us":71,"threads_started":1}
I20260809 15:38:29.302430 19887 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:38:29.301967 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.89.254:33497 (local address 127.3.89.195:42569)
0809 15:38:29.302283 (+ 316us) negotiation.cc:107] Waiting for socket to connect
0809 15:38:29.302337 (+ 54us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.89.254:33497: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":248,"thread_start_us":96,"threads_started":1}
I20260809 15:38:29.323698 19888 raft_consensus.cc:482] T f875addffcb84c74aab667cbd32c47b6 P b0a56d93c098468296a2000c9c5609ca [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 15:38:29.323864 19888 raft_consensus.cc:504] T f875addffcb84c74aab667cbd32c47b6 P b0a56d93c098468296a2000c9c5609ca [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "421cb337efc747e292ba53a026c44c02" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 40257 } } peers { permanent_uuid: "273a5fb47ef740c196b1a55c87ed7466" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 45639 } } peers { permanent_uuid: "b0a56d93c098468296a2000c9c5609ca" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 44707 } }
I20260809 15:38:29.324277 19888 leader_election.cc:288] T f875addffcb84c74aab667cbd32c47b6 P b0a56d93c098468296a2000c9c5609ca [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 421cb337efc747e292ba53a026c44c02 (127.3.89.195:40257), 273a5fb47ef740c196b1a55c87ed7466 (127.3.89.194:45639)
W20260809 15:38:29.325034 19438 leader_election.cc:341] T f875addffcb84c74aab667cbd32c47b6 P b0a56d93c098468296a2000c9c5609ca [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 421cb337efc747e292ba53a026c44c02 (127.3.89.195:40257): Not found: Tablet not found: f875addffcb84c74aab667cbd32c47b6
W20260809 15:38:29.325135 19440 leader_election.cc:341] T f875addffcb84c74aab667cbd32c47b6 P b0a56d93c098468296a2000c9c5609ca [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 273a5fb47ef740c196b1a55c87ed7466 (127.3.89.194:45639): Not found: Tablet not found: f875addffcb84c74aab667cbd32c47b6
I20260809 15:38:29.325266 19440 leader_election.cc:302] T f875addffcb84c74aab667cbd32c47b6 P b0a56d93c098468296a2000c9c5609ca [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b0a56d93c098468296a2000c9c5609ca; no voters: 273a5fb47ef740c196b1a55c87ed7466, 421cb337efc747e292ba53a026c44c02
I20260809 15:38:29.325439 19888 raft_consensus.cc:2726] T f875addffcb84c74aab667cbd32c47b6 P b0a56d93c098468296a2000c9c5609ca [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260809 15:38:29.982241 19889 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:38:29.981724 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.89.254:33497 (local address 127.3.89.193:49307)
0809 15:38:29.982084 (+ 360us) negotiation.cc:107] Waiting for socket to connect
0809 15:38:29.982144 (+ 60us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.89.254:33497: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":275,"thread_start_us":84,"threads_started":1}
I20260809 15:38:30.115356 19890 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:38:30.114953 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.89.254:33497 (local address 127.3.89.194:50999)
0809 15:38:30.115206 (+ 253us) negotiation.cc:107] Waiting for socket to connect
0809 15:38:30.115261 (+ 55us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.89.254:33497: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":184,"thread_start_us":73,"threads_started":1}
I20260809 15:38:30.303583 19891 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:38:30.303170 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.89.254:33497 (local address 127.3.89.195:55597)
0809 15:38:30.303429 (+ 259us) negotiation.cc:107] Waiting for socket to connect
0809 15:38:30.303496 (+ 67us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.89.254:33497: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":180,"thread_start_us":77,"threads_started":1}
I20260809 15:38:30.983556 19892 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:38:30.983080 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.89.254:33497 (local address 127.3.89.193:51043)
0809 15:38:30.983389 (+ 309us) negotiation.cc:107] Waiting for socket to connect
0809 15:38:30.983454 (+ 65us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.89.254:33497: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":229,"thread_start_us":93,"threads_started":1}
I20260809 15:38:31.116575 19893 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:38:31.116154 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.89.254:33497 (local address 127.3.89.194:38599)
0809 15:38:31.116421 (+ 267us) negotiation.cc:107] Waiting for socket to connect
0809 15:38:31.116476 (+ 55us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.89.254:33497: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":194,"thread_start_us":97,"threads_started":1}
I20260809 15:38:31.304740 19894 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:38:31.304316 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.89.254:33497 (local address 127.3.89.195:35407)
0809 15:38:31.304578 (+ 262us) negotiation.cc:107] Waiting for socket to connect
0809 15:38:31.304633 (+ 55us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.89.254:33497: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":192,"thread_start_us":96,"threads_started":1}
I20260809 15:38:31.984939 19895 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:38:31.984491 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.89.254:33497 (local address 127.3.89.193:34967)
0809 15:38:31.984781 (+ 290us) negotiation.cc:107] Waiting for socket to connect
0809 15:38:31.984842 (+ 61us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.89.254:33497: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":215,"thread_start_us":99,"threads_started":1}
I20260809 15:38:32.117794 19896 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:38:32.117379 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.89.254:33497 (local address 127.3.89.194:51335)
0809 15:38:32.117642 (+ 263us) negotiation.cc:107] Waiting for socket to connect
0809 15:38:32.117699 (+ 57us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.89.254:33497: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":192,"thread_start_us":98,"threads_started":1}
I20260809 15:38:32.305878 19897 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:38:32.305482 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.89.254:33497 (local address 127.3.89.195:59735)
0809 15:38:32.305732 (+ 250us) negotiation.cc:107] Waiting for socket to connect
0809 15:38:32.305785 (+ 53us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.89.254:33497: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":184,"thread_start_us":93,"threads_started":1}
I20260809 15:38:32.986255 19898 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:38:32.985781 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.89.254:33497 (local address 127.3.89.193:43423)
0809 15:38:32.986073 (+ 292us) negotiation.cc:107] Waiting for socket to connect
0809 15:38:32.986130 (+ 57us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.89.254:33497: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":193,"thread_start_us":75,"threads_started":1}
I20260809 15:38:33.118955 19899 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:38:33.118546 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.89.254:33497 (local address 127.3.89.194:53283)
0809 15:38:33.118804 (+ 258us) negotiation.cc:107] Waiting for socket to connect
0809 15:38:33.118861 (+ 57us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.89.254:33497: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":188,"thread_start_us":91,"threads_started":1}
I20260809 15:38:33.307035 19901 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:38:33.306573 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.89.254:33497 (local address 127.3.89.195:51135)
0809 15:38:33.306830 (+ 257us) negotiation.cc:107] Waiting for socket to connect
0809 15:38:33.306884 (+ 54us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.89.254:33497: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":190,"thread_start_us":96,"threads_started":1}
I20260809 15:38:33.402334 19902 raft_consensus.cc:482] T f875addffcb84c74aab667cbd32c47b6 P b0a56d93c098468296a2000c9c5609ca [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 15:38:33.402489 19902 raft_consensus.cc:504] T f875addffcb84c74aab667cbd32c47b6 P b0a56d93c098468296a2000c9c5609ca [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "421cb337efc747e292ba53a026c44c02" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 40257 } } peers { permanent_uuid: "273a5fb47ef740c196b1a55c87ed7466" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 45639 } } peers { permanent_uuid: "b0a56d93c098468296a2000c9c5609ca" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 44707 } }
I20260809 15:38:33.402915 19902 leader_election.cc:288] T f875addffcb84c74aab667cbd32c47b6 P b0a56d93c098468296a2000c9c5609ca [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 421cb337efc747e292ba53a026c44c02 (127.3.89.195:40257), 273a5fb47ef740c196b1a55c87ed7466 (127.3.89.194:45639)
W20260809 15:38:33.403589 19438 leader_election.cc:341] T f875addffcb84c74aab667cbd32c47b6 P b0a56d93c098468296a2000c9c5609ca [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 421cb337efc747e292ba53a026c44c02 (127.3.89.195:40257): Not found: Tablet not found: f875addffcb84c74aab667cbd32c47b6
W20260809 15:38:33.403682 19440 leader_election.cc:341] T f875addffcb84c74aab667cbd32c47b6 P b0a56d93c098468296a2000c9c5609ca [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 273a5fb47ef740c196b1a55c87ed7466 (127.3.89.194:45639): Not found: Tablet not found: f875addffcb84c74aab667cbd32c47b6
I20260809 15:38:33.403805 19440 leader_election.cc:302] T f875addffcb84c74aab667cbd32c47b6 P b0a56d93c098468296a2000c9c5609ca [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b0a56d93c098468296a2000c9c5609ca; no voters: 273a5fb47ef740c196b1a55c87ed7466, 421cb337efc747e292ba53a026c44c02
I20260809 15:38:33.403934 19902 raft_consensus.cc:2726] T f875addffcb84c74aab667cbd32c47b6 P b0a56d93c098468296a2000c9c5609ca [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260809 15:38:33.987546 19903 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:38:33.987138 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.89.254:33497 (local address 127.3.89.193:39033)
0809 15:38:33.987388 (+ 250us) negotiation.cc:107] Waiting for socket to connect
0809 15:38:33.987442 (+ 54us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.89.254:33497: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":180,"thread_start_us":83,"threads_started":1}
I20260809 15:38:34.120306 19904 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:38:34.119858 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.89.254:33497 (local address 127.3.89.194:59897)
0809 15:38:34.120139 (+ 281us) negotiation.cc:107] Waiting for socket to connect
0809 15:38:34.120198 (+ 59us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.89.254:33497: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":189,"thread_start_us":91,"threads_started":1}
I20260809 15:38:34.308235 19906 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:38:34.307784 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.89.254:33497 (local address 127.3.89.195:44801)
0809 15:38:34.308056 (+ 272us) negotiation.cc:107] Waiting for socket to connect
0809 15:38:34.308123 (+ 67us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.89.254:33497: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":189,"thread_start_us":79,"threads_started":1}
I20260809 15:38:34.988917 19907 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:38:34.988477 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.89.254:33497 (local address 127.3.89.193:52065)
0809 15:38:34.988757 (+ 280us) negotiation.cc:107] Waiting for socket to connect
0809 15:38:34.988821 (+ 64us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.89.254:33497: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":200,"thread_start_us":81,"threads_started":1}
I20260809 15:38:35.121394 19908 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:38:35.121013 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.89.254:33497 (local address 127.3.89.194:52157)
0809 15:38:35.121240 (+ 227us) negotiation.cc:107] Waiting for socket to connect
0809 15:38:35.121293 (+ 53us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.89.254:33497: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":162,"thread_start_us":72,"threads_started":1}
I20260809 15:38:35.309536 19909 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:38:35.309103 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.89.254:33497 (local address 127.3.89.195:55985)
0809 15:38:35.309383 (+ 280us) negotiation.cc:107] Waiting for socket to connect
0809 15:38:35.309440 (+ 57us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.89.254:33497: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":207,"thread_start_us":75,"threads_started":1}
I20260809 15:38:35.990140 19910 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:38:35.989743 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.89.254:33497 (local address 127.3.89.193:53063)
0809 15:38:35.989989 (+ 246us) negotiation.cc:107] Waiting for socket to connect
0809 15:38:35.990044 (+ 55us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.89.254:33497: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":176,"thread_start_us":79,"threads_started":1}
I20260809 15:38:36.122641 19911 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:38:36.122220 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.89.254:33497 (local address 127.3.89.194:48627)
0809 15:38:36.122488 (+ 268us) negotiation.cc:107] Waiting for socket to connect
0809 15:38:36.122544 (+ 56us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.89.254:33497: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":181,"thread_start_us":83,"threads_started":1}
I20260809 15:38:36.310680 19912 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:38:36.310278 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.89.254:33497 (local address 127.3.89.195:55539)
0809 15:38:36.310529 (+ 251us) negotiation.cc:107] Waiting for socket to connect
0809 15:38:36.310583 (+ 54us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.89.254:33497: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":183,"thread_start_us":79,"threads_started":1}
I20260809 15:38:36.991549 19913 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:38:36.991130 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.89.254:33497 (local address 127.3.89.193:59473)
0809 15:38:36.991394 (+ 264us) negotiation.cc:107] Waiting for socket to connect
0809 15:38:36.991451 (+ 57us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.89.254:33497: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":192,"thread_start_us":76,"threads_started":1}
I20260809 15:38:37.123939 19914 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:38:37.123541 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.89.254:33497 (local address 127.3.89.194:60021)
0809 15:38:37.123785 (+ 244us) negotiation.cc:107] Waiting for socket to connect
0809 15:38:37.123847 (+ 62us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.89.254:33497: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":168,"thread_start_us":93,"threads_started":1}
I20260809 15:38:37.300311 3431 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskm314YR/build/debug/bin/kudu with pid 19387
I20260809 15:38:37.310179 3431 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskm314YR/build/debug/bin/kudu with pid 19564
I20260809 15:38:37.312856 19915 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:38:37.312242 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.89.254:33497 (local address 127.3.89.195:46585)
0809 15:38:37.312710 (+ 468us) negotiation.cc:107] Waiting for socket to connect
0809 15:38:37.312763 (+ 53us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.89.254:33497: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":387,"thread_start_us":75,"threads_started":1}
I20260809 15:38:37.317720 3431 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskm314YR/build/debug/bin/kudu with pid 19713
2026-08-09T15:38:37Z chronyd exiting
[ OK ] SecurityITest.TestEnableAndDisableEncryptedIPKICert (19432 ms)
[ RUN ] SecurityITest.TestEnableAndDisableEncryptedTSK
Loading random data
Initializing database '/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786289732823074-3431-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 15:38:37 dist-test-slave-2w3w krb5kdc[19918](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 15:38:37 dist-test-slave-2w3w krb5kdc[19918](info): set up 2 sockets
Aug 09 15:38:37 dist-test-slave-2w3w krb5kdc[19918](info): commencing operation
krb5kdc: starting...
W20260809 15:38:38.405406 3431 mini_kdc.cc:121] Time spent starting KDC: real 1.061s user 0.000s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786289732823074-3431-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786289732823074-3431-0/krb5kdc/test-user.keytab.
Aug 09 15:38:38 dist-test-slave-2w3w krb5kdc[19918](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786289918, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T15:38:38Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T15:38:38Z Disabled control of system clock
WARNING: no policy specified for kudu/127.3.89.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.3.89.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.3.89.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786289732823074-3431-0/krb5kdc/kudu_127.3.89.254.keytab.
Entry for principal kudu/127.3.89.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786289732823074-3431-0/krb5kdc/kudu_127.3.89.254.keytab.
I20260809 15:38:38.531343 3431 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskm314YR/build/debug/bin/kudu
/tmp/dist-test-taskm314YR/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786289732823074-3431-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786289732823074-3431-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786289732823074-3431-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786289732823074-3431-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.3.89.254:41873
--webserver_interface=127.3.89.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786289732823074-3431-0/krb5kdc/kudu_127.3.89.254.keytab
--principal=kudu/127.3.89.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.3.89.212:37503
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.3.89.254:41873
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20 with env {KRB5CCNAME=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786289732823074-3431-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786289732823074-3431-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786289732823074-3431-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 15:38:38.643306 19934 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 15:38:38.643558 19934 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 15:38:38.643596 19934 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 15:38:38.647612 19934 flags.cc:425] Enabled experimental flag: --authn_token_validity_seconds=20
W20260809 15:38:38.647660 19934 flags.cc:425] Enabled experimental flag: --authz_token_validity_seconds=20
W20260809 15:38:38.647675 19934 flags.cc:425] Enabled experimental flag: --tsk_rotation_seconds=20
W20260809 15:38:38.647689 19934 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 15:38:38.647706 19934 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 15:38:38.647724 19934 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 15:38:38.647739 19934 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 15:38:38.647759 19934 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 15:38:38.652771 19934 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.3.89.212:37503
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786289732823074-3431-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786289732823074-3431-0/minicluster-data/master-0/wal
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20
--ipki_ca_key_size=768
--master_addresses=127.3.89.254:41873
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786289732823074-3431-0/krb5kdc/kudu_127.3.89.254.keytab
--principal=kudu/127.3.89.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.3.89.254:41873
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786289732823074-3431-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.3.89.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.19934
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786289732823074-3431-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 09 Aug 2026 14:46:55 UTC on ccycloud.quasar-yswkmf.root.comops.site
build id 4576540
I20260809 15:38:38.653707 19934 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 15:38:38.654636 19934 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 15:38:38.663738 19943 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 15:38:38.663748 19940 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 15:38:38.663748 19941 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 15:38:38.663792 19934 server_base.cc:1034] running on GCE node
I20260809 15:38:38.664642 19934 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 15:38:38.665748 19934 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 15:38:38.666937 19934 hybrid_clock.cc:648] HybridClock initialized: now 1786289918666923 us; error 29 us; skew 500 ppm
Aug 09 15:38:38 dist-test-slave-2w3w krb5kdc[19918](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786289918, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.89.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 15:38:38.674691 19934 init.cc:377] Logged in from keytab as kudu/127.3.89.254@KRBTEST.COM (short username kudu)
I20260809 15:38:38.675649 19934 webserver.cc:460] Webserver started at http://127.3.89.254:35627/ using document root <none> and password file <none>
I20260809 15:38:38.676198 19934 fs_manager.cc:359] Metadata directory not provided
I20260809 15:38:38.676259 19934 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 15:38:38.676429 19934 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 15:38:38.678434 19934 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786289732823074-3431-0/minicluster-data/master-0/data/instance:
uuid: "210f7703a6ee4761acec0985806eb015"
format_stamp: "Formatted at 2026-08-09 15:38:38 on dist-test-slave-2w3w"
server_key: "03ecf0ac536692172efc79bd865fd7d4"
server_key_iv: "397c712c73463366eb9a3a049bad8e83"
server_key_version: "encryptionkey@0"
I20260809 15:38:38.679028 19934 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786289732823074-3431-0/minicluster-data/master-0/wal/instance:
uuid: "210f7703a6ee4761acec0985806eb015"
format_stamp: "Formatted at 2026-08-09 15:38:38 on dist-test-slave-2w3w"
server_key: "03ecf0ac536692172efc79bd865fd7d4"
server_key_iv: "397c712c73463366eb9a3a049bad8e83"
server_key_version: "encryptionkey@0"
I20260809 15:38:38.682682 19934 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.002s sys 0.001s
I20260809 15:38:38.685079 19951 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 15:38:38.686293 19934 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260809 15:38:38.686424 19934 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786289732823074-3431-0/minicluster-data/master-0/data,/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786289732823074-3431-0/minicluster-data/master-0/wal
uuid: "210f7703a6ee4761acec0985806eb015"
format_stamp: "Formatted at 2026-08-09 15:38:38 on dist-test-slave-2w3w"
server_key: "03ecf0ac536692172efc79bd865fd7d4"
server_key_iv: "397c712c73463366eb9a3a049bad8e83"
server_key_version: "encryptionkey@0"
I20260809 15:38:38.686542 19934 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786289732823074-3431-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786289732823074-3431-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786289732823074-3431-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 15:38:38.698576 19934 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 15:38:38.701347 19934 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 15:38:38.701586 19934 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 15:38:38.708710 19934 rpc_server.cc:307] RPC server started. Bound to: 127.3.89.254:41873
I20260809 15:38:38.709018 20007 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.3.89.254:41873 every 8 connection(s)
I20260809 15:38:38.709689 19934 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786289732823074-3431-0/minicluster-data/master-0/data/info.pb
I20260809 15:38:38.712410 20010 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 15:38:38.717844 3431 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskm314YR/build/debug/bin/kudu as pid 19934
I20260809 15:38:38.717973 3431 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786289732823074-3431-0/minicluster-data/master-0/wal/instance
I20260809 15:38:38.718000 20010 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 210f7703a6ee4761acec0985806eb015: Bootstrap starting.
I20260809 15:38:38.718278 3431 external_mini_cluster.cc:1428] Setting key 29c6da86794cb83d04d65397ac75fdfe
I20260809 15:38:38.720609 20010 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 210f7703a6ee4761acec0985806eb015: Neither blocks nor log segments found. Creating new log.
I20260809 15:38:38.721405 20010 log.cc:824] T 00000000000000000000000000000000 P 210f7703a6ee4761acec0985806eb015: Log is configured to *not* fsync() on all Append() calls
I20260809 15:38:38.723428 20010 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 210f7703a6ee4761acec0985806eb015: No bootstrap required, opened a new log
Aug 09 15:38:38 dist-test-slave-2w3w krb5kdc[19918](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786289918, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.3.89.254@KRBTEST.COM
I20260809 15:38:38.727780 20010 raft_consensus.cc:348] T 00000000000000000000000000000000 P 210f7703a6ee4761acec0985806eb015 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "210f7703a6ee4761acec0985806eb015" member_type: VOTER last_known_addr { host: "127.3.89.254" port: 41873 } }
I20260809 15:38:38.728012 20010 raft_consensus.cc:374] T 00000000000000000000000000000000 P 210f7703a6ee4761acec0985806eb015 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 15:38:38.728084 20010 raft_consensus.cc:729] T 00000000000000000000000000000000 P 210f7703a6ee4761acec0985806eb015 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 210f7703a6ee4761acec0985806eb015, State: Initialized, Role: FOLLOWER
I20260809 15:38:38.728549 20010 consensus_queue.cc:260] T 00000000000000000000000000000000 P 210f7703a6ee4761acec0985806eb015 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "210f7703a6ee4761acec0985806eb015" member_type: VOTER last_known_addr { host: "127.3.89.254" port: 41873 } }
I20260809 15:38:38.728698 20010 raft_consensus.cc:388] T 00000000000000000000000000000000 P 210f7703a6ee4761acec0985806eb015 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 15:38:38.728780 20010 raft_consensus.cc:482] T 00000000000000000000000000000000 P 210f7703a6ee4761acec0985806eb015 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 15:38:38.728896 20010 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 210f7703a6ee4761acec0985806eb015 [term 0 FOLLOWER]: Advancing to term 1
I20260809 15:38:38.729787 20010 raft_consensus.cc:504] T 00000000000000000000000000000000 P 210f7703a6ee4761acec0985806eb015 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "210f7703a6ee4761acec0985806eb015" member_type: VOTER last_known_addr { host: "127.3.89.254" port: 41873 } }
I20260809 15:38:38.730116 20010 leader_election.cc:302] T 00000000000000000000000000000000 P 210f7703a6ee4761acec0985806eb015 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 210f7703a6ee4761acec0985806eb015; no voters:
I20260809 15:38:38.730505 20010 leader_election.cc:288] T 00000000000000000000000000000000 P 210f7703a6ee4761acec0985806eb015 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 15:38:38.730547 20015 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 210f7703a6ee4761acec0985806eb015 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 15:38:38.730762 20015 raft_consensus.cc:686] T 00000000000000000000000000000000 P 210f7703a6ee4761acec0985806eb015 [term 1 LEADER]: Becoming Leader. State: Replica: 210f7703a6ee4761acec0985806eb015, State: Running, Role: LEADER
I20260809 15:38:38.731145 20015 consensus_queue.cc:237] T 00000000000000000000000000000000 P 210f7703a6ee4761acec0985806eb015 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "210f7703a6ee4761acec0985806eb015" member_type: VOTER last_known_addr { host: "127.3.89.254" port: 41873 } }
I20260809 15:38:38.731835 20010 sys_catalog.cc:564] T 00000000000000000000000000000000 P 210f7703a6ee4761acec0985806eb015 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 15:38:38.732868 20017 sys_catalog.cc:455] T 00000000000000000000000000000000 P 210f7703a6ee4761acec0985806eb015 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "210f7703a6ee4761acec0985806eb015" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "210f7703a6ee4761acec0985806eb015" member_type: VOTER last_known_addr { host: "127.3.89.254" port: 41873 } } }
I20260809 15:38:38.733031 20017 sys_catalog.cc:458] T 00000000000000000000000000000000 P 210f7703a6ee4761acec0985806eb015 [sys.catalog]: This master's current role is: LEADER
I20260809 15:38:38.733434 20026 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 15:38:38.733417 20016 sys_catalog.cc:455] T 00000000000000000000000000000000 P 210f7703a6ee4761acec0985806eb015 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 210f7703a6ee4761acec0985806eb015. Latest consensus state: current_term: 1 leader_uuid: "210f7703a6ee4761acec0985806eb015" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "210f7703a6ee4761acec0985806eb015" member_type: VOTER last_known_addr { host: "127.3.89.254" port: 41873 } } }
I20260809 15:38:38.733603 20016 sys_catalog.cc:458] T 00000000000000000000000000000000 P 210f7703a6ee4761acec0985806eb015 [sys.catalog]: This master's current role is: LEADER
I20260809 15:38:38.733584 20013 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:38:38.719602 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54750 (local address 127.3.89.254:41873)
0809 15:38:38.719986 (+ 384us) server_negotiation.cc:206] Beginning negotiation
0809 15:38:38.720002 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0809 15:38:38.720037 (+ 35us) server_negotiation.cc:407] Connection header received
0809 15:38:38.720648 (+ 611us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 15:38:38.720690 (+ 42us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 15:38:38.721010 (+ 320us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 15:38:38.721286 (+ 276us) server_negotiation.cc:226] Negotiated authn=SASL
0809 15:38:38.722200 (+ 914us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:38:38.723247 (+ 1047us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:38:38.724418 (+ 1171us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:38:38.724879 (+ 461us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:38:38.725006 (+ 127us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:38:38.727885 (+ 2879us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 15:38:38.727890 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 15:38:38.727902 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 15:38:38.727929 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0809 15:38:38.729760 (+ 1831us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 15:38:38.730221 (+ 461us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 15:38:38.730227 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 15:38:38.730235 (+ 8us) server_negotiation.cc:967] calling sasl_server_step()
0809 15:38:38.730369 (+ 134us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 15:38:38.730715 (+ 346us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 15:38:38.730719 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 15:38:38.730720 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 15:38:38.731255 (+ 535us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 15:38:38.731993 (+ 738us) server_negotiation.cc:1035] Waiting for connection context
0809 15:38:38.732155 (+ 162us) server_negotiation.cc:299] Negotiation successful
0809 15:38:38.732483 (+ 328us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":242,"thread_start_us":126,"threads_started":1}
I20260809 15:38:38.737807 20026 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 15:38:38.743527 20026 catalog_manager.cc:1347] Generated new cluster ID: bc02bf064fd44d40b13be3238ff5b7dc
I20260809 15:38:38.743642 20026 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 15:38:38.787803 20026 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 15:38:38.789143 20026 catalog_manager.cc:1504] Loading token signing keys...
I20260809 15:38:38.805578 20026 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 210f7703a6ee4761acec0985806eb015: Generated new TSK 0
I20260809 15:38:38.806459 20026 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.3.89.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.3.89.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.3.89.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786289732823074-3431-0/krb5kdc/kudu_127.3.89.193.keytab.
Entry for principal kudu/127.3.89.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786289732823074-3431-0/krb5kdc/kudu_127.3.89.193.keytab.
I20260809 15:38:38.840237 3431 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskm314YR/build/debug/bin/kudu
/tmp/dist-test-taskm314YR/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786289732823074-3431-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786289732823074-3431-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786289732823074-3431-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786289732823074-3431-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.3.89.193:0
--local_ip_for_outbound_sockets=127.3.89.193
--webserver_interface=127.3.89.193
--webserver_port=0
--tserver_master_addrs=127.3.89.254:41873
--keytab_file=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786289732823074-3431-0/krb5kdc/kudu_127.3.89.193.keytab
--principal=kudu/127.3.89.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.3.89.212:37503
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786289732823074-3431-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786289732823074-3431-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786289732823074-3431-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 15:38:38.949295 20038 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 15:38:38.949581 20038 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 15:38:38.949656 20038 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 15:38:38.953598 20038 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 15:38:38.953675 20038 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 15:38:38.953758 20038 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.3.89.193
I20260809 15:38:38.959236 20038 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.3.89.212:37503
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786289732823074-3431-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786289732823074-3431-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786289732823074-3431-0/krb5kdc/kudu_127.3.89.193.keytab
--principal=kudu/127.3.89.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.3.89.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786289732823074-3431-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.3.89.193
--webserver_port=0
--tserver_master_addrs=127.3.89.254:41873
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.20038
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.3.89.193
--log_dir=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786289732823074-3431-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 09 Aug 2026 14:46:55 UTC on ccycloud.quasar-yswkmf.root.comops.site
build id 4576540
I20260809 15:38:38.960273 20038 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 15:38:38.961254 20038 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 15:38:38.971069 20044 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 15:38:38.971079 20043 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 15:38:38.971055 20046 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 15:38:38.971334 20038 server_base.cc:1034] running on GCE node
I20260809 15:38:38.971865 20038 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 15:38:38.972499 20038 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 15:38:38.973693 20038 hybrid_clock.cc:648] HybridClock initialized: now 1786289918973661 us; error 46 us; skew 500 ppm
Aug 09 15:38:38 dist-test-slave-2w3w krb5kdc[19918](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786289918, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.89.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 15:38:38.976658 20038 init.cc:377] Logged in from keytab as kudu/127.3.89.193@KRBTEST.COM (short username kudu)
I20260809 15:38:38.977536 20038 webserver.cc:460] Webserver started at http://127.3.89.193:44001/ using document root <none> and password file <none>
I20260809 15:38:38.978096 20038 fs_manager.cc:359] Metadata directory not provided
I20260809 15:38:38.978188 20038 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 15:38:38.978420 20038 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 15:38:38.980512 20038 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786289732823074-3431-0/minicluster-data/ts-0/data/instance:
uuid: "23bd656811574169affd015915dd21e5"
format_stamp: "Formatted at 2026-08-09 15:38:38 on dist-test-slave-2w3w"
server_key: "a4aab3d472b4637e64f780d2befe66ff"
server_key_iv: "8968b27898f895919b075309b448b735"
server_key_version: "encryptionkey@0"
I20260809 15:38:38.981083 20038 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786289732823074-3431-0/minicluster-data/ts-0/wal/instance:
uuid: "23bd656811574169affd015915dd21e5"
format_stamp: "Formatted at 2026-08-09 15:38:38 on dist-test-slave-2w3w"
server_key: "a4aab3d472b4637e64f780d2befe66ff"
server_key_iv: "8968b27898f895919b075309b448b735"
server_key_version: "encryptionkey@0"
I20260809 15:38:38.984786 20038 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.005s sys 0.000s
I20260809 15:38:38.987213 20054 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 15:38:38.988688 20038 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260809 15:38:38.988804 20038 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786289732823074-3431-0/minicluster-data/ts-0/data,/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786289732823074-3431-0/minicluster-data/ts-0/wal
uuid: "23bd656811574169affd015915dd21e5"
format_stamp: "Formatted at 2026-08-09 15:38:38 on dist-test-slave-2w3w"
server_key: "a4aab3d472b4637e64f780d2befe66ff"
server_key_iv: "8968b27898f895919b075309b448b735"
server_key_version: "encryptionkey@0"
I20260809 15:38:38.988917 20038 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786289732823074-3431-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786289732823074-3431-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786289732823074-3431-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 15:38:39.003556 20038 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 15:38:39.006232 20038 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 15:38:39.006390 20038 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 15:38:39.006899 20038 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 15:38:39.007741 20038 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 15:38:39.007791 20038 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 15:38:39.007830 20038 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 15:38:39.007846 20038 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 15:38:39.016494 20038 rpc_server.cc:307] RPC server started. Bound to: 127.3.89.193:36955
I20260809 15:38:39.016562 20178 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.3.89.193:36955 every 8 connection(s)
I20260809 15:38:39.017414 20038 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786289732823074-3431-0/minicluster-data/ts-0/data/info.pb
Aug 09 15:38:39 dist-test-slave-2w3w krb5kdc[19918](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786289918, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.89.193@KRBTEST.COM for kudu/127.3.89.254@KRBTEST.COM
I20260809 15:38:39.027093 3431 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskm314YR/build/debug/bin/kudu as pid 20038
I20260809 15:38:39.027222 3431 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786289732823074-3431-0/minicluster-data/ts-0/wal/instance
I20260809 15:38:39.027490 3431 external_mini_cluster.cc:1428] Setting key 8e8099fe589e49544eddaaf894d44cd5
I20260809 15:38:39.030932 20013 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:38:39.019194 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.89.193:57627 (local address 127.3.89.254:41873)
0809 15:38:39.019326 (+ 132us) server_negotiation.cc:206] Beginning negotiation
0809 15:38:39.019330 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 15:38:39.020331 (+ 1001us) server_negotiation.cc:407] Connection header received
0809 15:38:39.021208 (+ 877us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 15:38:39.021212 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 15:38:39.021274 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 15:38:39.021381 (+ 107us) server_negotiation.cc:226] Negotiated authn=SASL
0809 15:38:39.022499 (+ 1118us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:38:39.023139 (+ 640us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:38:39.024283 (+ 1144us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:38:39.024529 (+ 246us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:38:39.024640 (+ 111us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:38:39.027082 (+ 2442us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 15:38:39.027085 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 15:38:39.027087 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 15:38:39.027112 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0809 15:38:39.028751 (+ 1639us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 15:38:39.029388 (+ 637us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 15:38:39.029391 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 15:38:39.029392 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 15:38:39.029473 (+ 81us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 15:38:39.029855 (+ 382us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 15:38:39.029857 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 15:38:39.029859 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 15:38:39.030023 (+ 164us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 15:38:39.030097 (+ 74us) server_negotiation.cc:1035] Waiting for connection context
0809 15:38:39.030706 (+ 609us) server_negotiation.cc:299] Negotiation successful
0809 15:38:39.030811 (+ 105us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":44}
I20260809 15:38:39.031950 20181 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:38:39.019535 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.89.254:41873 (local address 127.3.89.193:57627)
0809 15:38:39.020184 (+ 649us) negotiation.cc:107] Waiting for socket to connect
0809 15:38:39.020224 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0809 15:38:39.021005 (+ 781us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 15:38:39.021545 (+ 540us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 15:38:39.021551 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 15:38:39.021890 (+ 339us) client_negotiation.cc:189] Negotiated authn=SASL
0809 15:38:39.022344 (+ 454us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:38:39.022356 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:38:39.023268 (+ 912us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:38:39.023271 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:38:39.024144 (+ 873us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:38:39.024153 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:38:39.024676 (+ 523us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:38:39.024680 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:38:39.024852 (+ 172us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:38:39.025373 (+ 521us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 15:38:39.025389 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0809 15:38:39.026857 (+ 1468us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 15:38:39.028930 (+ 2073us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 15:38:39.028935 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 15:38:39.028948 (+ 13us) client_negotiation.cc:760] calling sasl_client_step()
0809 15:38:39.029288 (+ 340us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 15:38:39.029569 (+ 281us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 15:38:39.029575 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 15:38:39.029579 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0809 15:38:39.029765 (+ 186us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 15:38:39.030125 (+ 360us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 15:38:39.030134 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 15:38:39.030450 (+ 316us) client_negotiation.cc:769] Sending connection context
0809 15:38:39.030663 (+ 213us) client_negotiation.cc:240] Negotiation successful
0809 15:38:39.031135 (+ 472us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":472,"thread_start_us":120,"threads_started":1}
I20260809 15:38:39.033301 20179 heartbeater.cc:347] Connected to a master server at 127.3.89.254:41873
I20260809 15:38:39.033537 20179 heartbeater.cc:464] Registering TS with master...
I20260809 15:38:39.034089 20179 heartbeater.cc:511] Master 127.3.89.254:41873 requested a full tablet report, sending...
I20260809 15:38:39.035645 19968 ts_manager.cc:194] Registered new tserver with Master: 23bd656811574169affd015915dd21e5 (127.3.89.193:36955)
I20260809 15:38:39.037256 19968 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.3.89.193@KRBTEST.COM'} at 127.3.89.193:57627
WARNING: no policy specified for kudu/127.3.89.194@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.3.89.194@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.3.89.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786289732823074-3431-0/krb5kdc/kudu_127.3.89.194.keytab.
Entry for principal kudu/127.3.89.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786289732823074-3431-0/krb5kdc/kudu_127.3.89.194.keytab.
I20260809 15:38:39.058017 3431 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskm314YR/build/debug/bin/kudu
/tmp/dist-test-taskm314YR/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786289732823074-3431-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786289732823074-3431-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786289732823074-3431-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786289732823074-3431-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.3.89.194:0
--local_ip_for_outbound_sockets=127.3.89.194
--webserver_interface=127.3.89.194
--webserver_port=0
--tserver_master_addrs=127.3.89.254:41873
--keytab_file=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786289732823074-3431-0/krb5kdc/kudu_127.3.89.194.keytab
--principal=kudu/127.3.89.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.3.89.212:37503
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786289732823074-3431-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786289732823074-3431-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786289732823074-3431-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 15:38:39.168517 20184 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 15:38:39.168783 20184 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 15:38:39.168856 20184 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 15:38:39.172978 20184 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 15:38:39.173058 20184 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 15:38:39.173152 20184 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.3.89.194
I20260809 15:38:39.178315 20184 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.3.89.212:37503
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786289732823074-3431-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786289732823074-3431-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786289732823074-3431-0/krb5kdc/kudu_127.3.89.194.keytab
--principal=kudu/127.3.89.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.3.89.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786289732823074-3431-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.3.89.194
--webserver_port=0
--tserver_master_addrs=127.3.89.254:41873
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.20184
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.3.89.194
--log_dir=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786289732823074-3431-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 09 Aug 2026 14:46:55 UTC on ccycloud.quasar-yswkmf.root.comops.site
build id 4576540
I20260809 15:38:39.179404 20184 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 15:38:39.180449 20184 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 15:38:39.189870 20190 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 15:38:39.189942 20192 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 15:38:39.189836 20189 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 15:38:39.190064 20184 server_base.cc:1034] running on GCE node
I20260809 15:38:39.190599 20184 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 15:38:39.191267 20184 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 15:38:39.192411 20184 hybrid_clock.cc:648] HybridClock initialized: now 1786289919192386 us; error 36 us; skew 500 ppm
Aug 09 15:38:39 dist-test-slave-2w3w krb5kdc[19918](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786289919, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.89.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 15:38:39.195325 20184 init.cc:377] Logged in from keytab as kudu/127.3.89.194@KRBTEST.COM (short username kudu)
I20260809 15:38:39.196123 20184 webserver.cc:460] Webserver started at http://127.3.89.194:39463/ using document root <none> and password file <none>
I20260809 15:38:39.196641 20184 fs_manager.cc:359] Metadata directory not provided
I20260809 15:38:39.196700 20184 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 15:38:39.196862 20184 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 15:38:39.198843 20184 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786289732823074-3431-0/minicluster-data/ts-1/data/instance:
uuid: "7ba1b776d4734e909008ca7c997fde5f"
format_stamp: "Formatted at 2026-08-09 15:38:39 on dist-test-slave-2w3w"
server_key: "05ddc3e034248186ff04a1599e9b0be6"
server_key_iv: "ac5b212a1983e3e9d8ae56c81442b5d1"
server_key_version: "encryptionkey@0"
I20260809 15:38:39.199489 20184 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786289732823074-3431-0/minicluster-data/ts-1/wal/instance:
uuid: "7ba1b776d4734e909008ca7c997fde5f"
format_stamp: "Formatted at 2026-08-09 15:38:39 on dist-test-slave-2w3w"
server_key: "05ddc3e034248186ff04a1599e9b0be6"
server_key_iv: "ac5b212a1983e3e9d8ae56c81442b5d1"
server_key_version: "encryptionkey@0"
I20260809 15:38:39.203519 20184 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.001s
I20260809 15:38:39.205832 20200 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 15:38:39.207010 20184 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260809 15:38:39.207141 20184 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786289732823074-3431-0/minicluster-data/ts-1/data,/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786289732823074-3431-0/minicluster-data/ts-1/wal
uuid: "7ba1b776d4734e909008ca7c997fde5f"
format_stamp: "Formatted at 2026-08-09 15:38:39 on dist-test-slave-2w3w"
server_key: "05ddc3e034248186ff04a1599e9b0be6"
server_key_iv: "ac5b212a1983e3e9d8ae56c81442b5d1"
server_key_version: "encryptionkey@0"
I20260809 15:38:39.207253 20184 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786289732823074-3431-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786289732823074-3431-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786289732823074-3431-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 15:38:39.235062 20184 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 15:38:39.237905 20184 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 15:38:39.238112 20184 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 15:38:39.238754 20184 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 15:38:39.239594 20184 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 15:38:39.239643 20184 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 15:38:39.239683 20184 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 15:38:39.239704 20184 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 15:38:39.248492 20184 rpc_server.cc:307] RPC server started. Bound to: 127.3.89.194:32773
I20260809 15:38:39.248512 20321 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.3.89.194:32773 every 8 connection(s)
I20260809 15:38:39.249366 20184 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786289732823074-3431-0/minicluster-data/ts-1/data/info.pb
I20260809 15:38:39.254561 3431 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskm314YR/build/debug/bin/kudu as pid 20184
I20260809 15:38:39.254689 3431 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786289732823074-3431-0/minicluster-data/ts-1/wal/instance
I20260809 15:38:39.254943 3431 external_mini_cluster.cc:1428] Setting key 2ff7e9ca1e0eabacd52e8b73b4b121cc
Aug 09 15:38:39 dist-test-slave-2w3w krb5kdc[19918](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786289919, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.89.194@KRBTEST.COM for kudu/127.3.89.254@KRBTEST.COM
I20260809 15:38:39.263068 20013 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:38:39.251102 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.89.194:56811 (local address 127.3.89.254:41873)
0809 15:38:39.251247 (+ 145us) server_negotiation.cc:206] Beginning negotiation
0809 15:38:39.251252 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 15:38:39.252078 (+ 826us) server_negotiation.cc:407] Connection header received
0809 15:38:39.253329 (+ 1251us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 15:38:39.253333 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 15:38:39.253390 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 15:38:39.253485 (+ 95us) server_negotiation.cc:226] Negotiated authn=SASL
0809 15:38:39.254648 (+ 1163us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:38:39.255343 (+ 695us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:38:39.256673 (+ 1330us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:38:39.256906 (+ 233us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:38:39.257015 (+ 109us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:38:39.259576 (+ 2561us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 15:38:39.259579 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 15:38:39.259581 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 15:38:39.259604 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0809 15:38:39.261048 (+ 1444us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 15:38:39.261549 (+ 501us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 15:38:39.261554 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 15:38:39.261556 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 15:38:39.261637 (+ 81us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 15:38:39.262004 (+ 367us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 15:38:39.262008 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 15:38:39.262009 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 15:38:39.262183 (+ 174us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 15:38:39.262287 (+ 104us) server_negotiation.cc:1035] Waiting for connection context
0809 15:38:39.262747 (+ 460us) server_negotiation.cc:299] Negotiation successful
0809 15:38:39.262870 (+ 123us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":49}
I20260809 15:38:39.263830 20324 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:38:39.251502 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.89.254:41873 (local address 127.3.89.194:56811)
0809 15:38:39.251933 (+ 431us) negotiation.cc:107] Waiting for socket to connect
0809 15:38:39.251976 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0809 15:38:39.253125 (+ 1149us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 15:38:39.253653 (+ 528us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 15:38:39.253660 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 15:38:39.254057 (+ 397us) client_negotiation.cc:189] Negotiated authn=SASL
0809 15:38:39.254509 (+ 452us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:38:39.254522 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:38:39.255469 (+ 947us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:38:39.255477 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:38:39.256550 (+ 1073us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:38:39.256559 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:38:39.257052 (+ 493us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:38:39.257056 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:38:39.257217 (+ 161us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:38:39.257796 (+ 579us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 15:38:39.257812 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0809 15:38:39.259410 (+ 1598us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 15:38:39.261169 (+ 1759us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 15:38:39.261175 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 15:38:39.261188 (+ 13us) client_negotiation.cc:760] calling sasl_client_step()
0809 15:38:39.261447 (+ 259us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 15:38:39.261753 (+ 306us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 15:38:39.261758 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 15:38:39.261759 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0809 15:38:39.261893 (+ 134us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 15:38:39.262288 (+ 395us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 15:38:39.262295 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 15:38:39.262527 (+ 232us) client_negotiation.cc:769] Sending connection context
0809 15:38:39.262703 (+ 176us) client_negotiation.cc:240] Negotiation successful
0809 15:38:39.263041 (+ 338us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":248,"thread_start_us":105,"threads_started":1}
I20260809 15:38:39.265170 20322 heartbeater.cc:347] Connected to a master server at 127.3.89.254:41873
I20260809 15:38:39.265399 20322 heartbeater.cc:464] Registering TS with master...
WARNING: no policy specified for kudu/127.3.89.195@KRBTEST.COM; defaulting to no policy
I20260809 15:38:39.265872 20322 heartbeater.cc:511] Master 127.3.89.254:41873 requested a full tablet report, sending...
I20260809 15:38:39.266865 19968 ts_manager.cc:194] Registered new tserver with Master: 7ba1b776d4734e909008ca7c997fde5f (127.3.89.194:32773)
I20260809 15:38:39.267444 19968 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.3.89.194@KRBTEST.COM'} at 127.3.89.194:56811
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.3.89.195@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.3.89.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786289732823074-3431-0/krb5kdc/kudu_127.3.89.195.keytab.
Entry for principal kudu/127.3.89.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786289732823074-3431-0/krb5kdc/kudu_127.3.89.195.keytab.
I20260809 15:38:39.285611 3431 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskm314YR/build/debug/bin/kudu
/tmp/dist-test-taskm314YR/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786289732823074-3431-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786289732823074-3431-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786289732823074-3431-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786289732823074-3431-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.3.89.195:0
--local_ip_for_outbound_sockets=127.3.89.195
--webserver_interface=127.3.89.195
--webserver_port=0
--tserver_master_addrs=127.3.89.254:41873
--keytab_file=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786289732823074-3431-0/krb5kdc/kudu_127.3.89.195.keytab
--principal=kudu/127.3.89.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.3.89.212:37503
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786289732823074-3431-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786289732823074-3431-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786289732823074-3431-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 15:38:39.396396 20327 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 15:38:39.396623 20327 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 15:38:39.396664 20327 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 15:38:39.400498 20327 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 15:38:39.400544 20327 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 15:38:39.400599 20327 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.3.89.195
I20260809 15:38:39.405546 20327 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.3.89.212:37503
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786289732823074-3431-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786289732823074-3431-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786289732823074-3431-0/krb5kdc/kudu_127.3.89.195.keytab
--principal=kudu/127.3.89.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.3.89.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786289732823074-3431-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.3.89.195
--webserver_port=0
--tserver_master_addrs=127.3.89.254:41873
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.20327
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.3.89.195
--log_dir=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786289732823074-3431-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 09 Aug 2026 14:46:55 UTC on ccycloud.quasar-yswkmf.root.comops.site
build id 4576540
I20260809 15:38:39.406654 20327 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 15:38:39.407694 20327 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 15:38:39.417119 20335 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 15:38:39.417198 20327 server_base.cc:1034] running on GCE node
W20260809 15:38:39.417124 20333 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 15:38:39.417128 20332 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 15:38:39.417838 20327 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 15:38:39.418432 20327 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 15:38:39.419629 20327 hybrid_clock.cc:648] HybridClock initialized: now 1786289919419599 us; error 40 us; skew 500 ppm
Aug 09 15:38:39 dist-test-slave-2w3w krb5kdc[19918](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786289919, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.89.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 15:38:39.422379 20327 init.cc:377] Logged in from keytab as kudu/127.3.89.195@KRBTEST.COM (short username kudu)
I20260809 15:38:39.423277 20327 webserver.cc:460] Webserver started at http://127.3.89.195:43913/ using document root <none> and password file <none>
I20260809 15:38:39.423821 20327 fs_manager.cc:359] Metadata directory not provided
I20260809 15:38:39.423909 20327 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 15:38:39.424124 20327 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 15:38:39.426098 20327 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786289732823074-3431-0/minicluster-data/ts-2/data/instance:
uuid: "440d47534a314ca88247d80185c766f8"
format_stamp: "Formatted at 2026-08-09 15:38:39 on dist-test-slave-2w3w"
server_key: "62d7b19b3c4c4511cb8d34b326b0568e"
server_key_iv: "1f6e4f692caf051f3d781a8ccd66e70b"
server_key_version: "encryptionkey@0"
I20260809 15:38:39.426654 20327 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786289732823074-3431-0/minicluster-data/ts-2/wal/instance:
uuid: "440d47534a314ca88247d80185c766f8"
format_stamp: "Formatted at 2026-08-09 15:38:39 on dist-test-slave-2w3w"
server_key: "62d7b19b3c4c4511cb8d34b326b0568e"
server_key_iv: "1f6e4f692caf051f3d781a8ccd66e70b"
server_key_version: "encryptionkey@0"
I20260809 15:38:39.430341 20327 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.006s sys 0.000s
I20260809 15:38:39.432869 20342 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 15:38:39.434070 20327 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.000s sys 0.002s
I20260809 15:38:39.434197 20327 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786289732823074-3431-0/minicluster-data/ts-2/data,/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786289732823074-3431-0/minicluster-data/ts-2/wal
uuid: "440d47534a314ca88247d80185c766f8"
format_stamp: "Formatted at 2026-08-09 15:38:39 on dist-test-slave-2w3w"
server_key: "62d7b19b3c4c4511cb8d34b326b0568e"
server_key_iv: "1f6e4f692caf051f3d781a8ccd66e70b"
server_key_version: "encryptionkey@0"
I20260809 15:38:39.434317 20327 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786289732823074-3431-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786289732823074-3431-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786289732823074-3431-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 15:38:39.455808 20327 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 15:38:39.459225 20327 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 15:38:39.459434 20327 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 15:38:39.459973 20327 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 15:38:39.460729 20327 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 15:38:39.460776 20327 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 15:38:39.460819 20327 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 15:38:39.460840 20327 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 15:38:39.469877 20327 rpc_server.cc:307] RPC server started. Bound to: 127.3.89.195:44003
I20260809 15:38:39.469907 20466 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.3.89.195:44003 every 8 connection(s)
I20260809 15:38:39.470752 20327 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786289732823074-3431-0/minicluster-data/ts-2/data/info.pb
I20260809 15:38:39.472213 3431 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskm314YR/build/debug/bin/kudu as pid 20327
I20260809 15:38:39.472344 3431 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786289732823074-3431-0/minicluster-data/ts-2/wal/instance
I20260809 15:38:39.472663 3431 external_mini_cluster.cc:1428] Setting key 48fd9bb116666f3be1a71e990c9a7ca4
Aug 09 15:38:39 dist-test-slave-2w3w krb5kdc[19918](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786289919, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.89.195@KRBTEST.COM for kudu/127.3.89.254@KRBTEST.COM
I20260809 15:38:39.483994 20013 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:38:39.472554 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.89.195:51027 (local address 127.3.89.254:41873)
0809 15:38:39.472709 (+ 155us) server_negotiation.cc:206] Beginning negotiation
0809 15:38:39.472715 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 15:38:39.474200 (+ 1485us) server_negotiation.cc:407] Connection header received
0809 15:38:39.475312 (+ 1112us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 15:38:39.475315 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 15:38:39.475361 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 15:38:39.475429 (+ 68us) server_negotiation.cc:226] Negotiated authn=SASL
0809 15:38:39.476555 (+ 1126us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:38:39.477090 (+ 535us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:38:39.478189 (+ 1099us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:38:39.478409 (+ 220us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:38:39.478485 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:38:39.480865 (+ 2380us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 15:38:39.480867 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 15:38:39.480869 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 15:38:39.480892 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0809 15:38:39.482289 (+ 1397us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 15:38:39.482704 (+ 415us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 15:38:39.482707 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 15:38:39.482708 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 15:38:39.482778 (+ 70us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 15:38:39.483100 (+ 322us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 15:38:39.483103 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 15:38:39.483104 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 15:38:39.483264 (+ 160us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 15:38:39.483349 (+ 85us) server_negotiation.cc:1035] Waiting for connection context
0809 15:38:39.483768 (+ 419us) server_negotiation.cc:299] Negotiation successful
0809 15:38:39.483877 (+ 109us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":55}
I20260809 15:38:39.484745 20469 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:38:39.472938 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.89.254:41873 (local address 127.3.89.195:51027)
0809 15:38:39.474030 (+ 1092us) negotiation.cc:107] Waiting for socket to connect
0809 15:38:39.474095 (+ 65us) client_negotiation.cc:174] Beginning negotiation
0809 15:38:39.475116 (+ 1021us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 15:38:39.475589 (+ 473us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 15:38:39.475596 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 15:38:39.475929 (+ 333us) client_negotiation.cc:189] Negotiated authn=SASL
0809 15:38:39.476412 (+ 483us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:38:39.476424 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:38:39.477194 (+ 770us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:38:39.477197 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:38:39.478085 (+ 888us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:38:39.478094 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:38:39.478537 (+ 443us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:38:39.478543 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:38:39.478682 (+ 139us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:38:39.479221 (+ 539us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 15:38:39.479237 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0809 15:38:39.480703 (+ 1466us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 15:38:39.482378 (+ 1675us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 15:38:39.482381 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 15:38:39.482390 (+ 9us) client_negotiation.cc:760] calling sasl_client_step()
0809 15:38:39.482624 (+ 234us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 15:38:39.482851 (+ 227us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 15:38:39.482853 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 15:38:39.482854 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0809 15:38:39.483026 (+ 172us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 15:38:39.483355 (+ 329us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 15:38:39.483360 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 15:38:39.483560 (+ 200us) client_negotiation.cc:769] Sending connection context
0809 15:38:39.483699 (+ 139us) client_negotiation.cc:240] Negotiation successful
0809 15:38:39.483922 (+ 223us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":878,"thread_start_us":730,"threads_started":1}
I20260809 15:38:39.486002 20467 heartbeater.cc:347] Connected to a master server at 127.3.89.254:41873
I20260809 15:38:39.486244 20467 heartbeater.cc:464] Registering TS with master...
I20260809 15:38:39.486702 20467 heartbeater.cc:511] Master 127.3.89.254:41873 requested a full tablet report, sending...
I20260809 15:38:39.487681 19968 ts_manager.cc:194] Registered new tserver with Master: 440d47534a314ca88247d80185c766f8 (127.3.89.195:44003)
I20260809 15:38:39.488181 19968 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.3.89.195@KRBTEST.COM'} at 127.3.89.195:51027
I20260809 15:38:39.498181 3431 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260809 15:38:39.507498 20013 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:38:39.500186 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54754 (local address 127.3.89.254:41873)
0809 15:38:39.500299 (+ 113us) server_negotiation.cc:206] Beginning negotiation
0809 15:38:39.500302 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0809 15:38:39.500502 (+ 200us) server_negotiation.cc:407] Connection header received
0809 15:38:39.500597 (+ 95us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 15:38:39.500599 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 15:38:39.500636 (+ 37us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 15:38:39.500698 (+ 62us) server_negotiation.cc:226] Negotiated authn=SASL
0809 15:38:39.501356 (+ 658us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:38:39.501876 (+ 520us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:38:39.502772 (+ 896us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:38:39.503087 (+ 315us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:38:39.503184 (+ 97us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:38:39.504436 (+ 1252us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 15:38:39.504439 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 15:38:39.504441 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 15:38:39.504465 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0809 15:38:39.506036 (+ 1571us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 15:38:39.506406 (+ 370us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 15:38:39.506410 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 15:38:39.506411 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 15:38:39.506488 (+ 77us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 15:38:39.506706 (+ 218us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 15:38:39.506708 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 15:38:39.506710 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 15:38:39.506927 (+ 217us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 15:38:39.507116 (+ 189us) server_negotiation.cc:1035] Waiting for connection context
0809 15:38:39.507297 (+ 181us) server_negotiation.cc:299] Negotiation successful
0809 15:38:39.507390 (+ 93us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":38}
I20260809 15:38:39.510092 19968 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:54754:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260809 15:38:39.513592 19968 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260809 15:38:39.528635 20480 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:38:39.522300 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45998 (local address 127.3.89.195:44003)
0809 15:38:39.523200 (+ 900us) server_negotiation.cc:206] Beginning negotiation
0809 15:38:39.523216 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0809 15:38:39.523255 (+ 39us) server_negotiation.cc:407] Connection header received
0809 15:38:39.523338 (+ 83us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 15:38:39.523345 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 15:38:39.523584 (+ 239us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 15:38:39.524140 (+ 556us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 15:38:39.524323 (+ 183us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:38:39.525283 (+ 960us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:38:39.527281 (+ 1998us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:38:39.527875 (+ 594us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:38:39.528006 (+ 131us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:38:39.528161 (+ 155us) server_negotiation.cc:1035] Waiting for connection context
0809 15:38:39.528359 (+ 198us) server_negotiation.cc:299] Negotiation successful
0809 15:38:39.528463 (+ 104us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":795,"thread_start_us":192,"threads_started":1}
I20260809 15:38:39.528625 20478 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:38:39.522136 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.89.195:44003 (local address 127.0.0.1:45998)
0809 15:38:39.522670 (+ 534us) negotiation.cc:107] Waiting for socket to connect
0809 15:38:39.522724 (+ 54us) client_negotiation.cc:174] Beginning negotiation
0809 15:38:39.522862 (+ 138us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 15:38:39.523778 (+ 916us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 15:38:39.523784 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 15:38:39.523815 (+ 31us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 15:38:39.523951 (+ 136us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:38:39.523956 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:38:39.525412 (+ 1456us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:38:39.525419 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:38:39.527132 (+ 1713us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:38:39.527144 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:38:39.528007 (+ 863us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:38:39.528011 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:38:39.528129 (+ 118us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:38:39.528168 (+ 39us) client_negotiation.cc:769] Sending connection context
0809 15:38:39.528310 (+ 142us) client_negotiation.cc:240] Negotiation successful
0809 15:38:39.528482 (+ 172us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":357,"mutex_wait_us":55,"thread_start_us":73,"threads_started":1}
I20260809 15:38:39.528995 20479 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:38:39.522829 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.89.194:32773 (local address 127.0.0.1:34344)
0809 15:38:39.523680 (+ 851us) negotiation.cc:107] Waiting for socket to connect
0809 15:38:39.523693 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0809 15:38:39.523778 (+ 85us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 15:38:39.524464 (+ 686us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 15:38:39.524467 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 15:38:39.524487 (+ 20us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 15:38:39.524591 (+ 104us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:38:39.524598 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:38:39.525905 (+ 1307us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:38:39.525911 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:38:39.527070 (+ 1159us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:38:39.527081 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:38:39.528336 (+ 1255us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:38:39.528358 (+ 22us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:38:39.528649 (+ 291us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:38:39.528693 (+ 44us) client_negotiation.cc:769] Sending connection context
0809 15:38:39.528776 (+ 83us) client_negotiation.cc:240] Negotiation successful
0809 15:38:39.528856 (+ 80us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":791,"mutex_wait_us":137,"thread_start_us":92,"threads_started":1}
I20260809 15:38:39.529523 20482 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:38:39.523011 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34344 (local address 127.3.89.194:32773)
0809 15:38:39.524031 (+ 1020us) server_negotiation.cc:206] Beginning negotiation
0809 15:38:39.524042 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0809 15:38:39.524063 (+ 21us) server_negotiation.cc:407] Connection header received
0809 15:38:39.524135 (+ 72us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 15:38:39.524139 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 15:38:39.524316 (+ 177us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 15:38:39.524478 (+ 162us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 15:38:39.524735 (+ 257us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:38:39.525763 (+ 1028us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:38:39.527224 (+ 1461us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:38:39.528045 (+ 821us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:38:39.528142 (+ 97us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:38:39.528270 (+ 128us) server_negotiation.cc:1035] Waiting for connection context
0809 15:38:39.529267 (+ 997us) server_negotiation.cc:299] Negotiation successful
0809 15:38:39.529379 (+ 112us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":906,"thread_start_us":98,"threads_started":1}
I20260809 15:38:39.529659 20481 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:38:39.522266 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.89.193:36955 (local address 127.0.0.1:33094)
0809 15:38:39.523174 (+ 908us) negotiation.cc:107] Waiting for socket to connect
0809 15:38:39.523188 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0809 15:38:39.523281 (+ 93us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 15:38:39.524049 (+ 768us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 15:38:39.524055 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 15:38:39.524079 (+ 24us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 15:38:39.524158 (+ 79us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:38:39.524164 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:38:39.525618 (+ 1454us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:38:39.525622 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:38:39.527052 (+ 1430us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:38:39.527065 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:38:39.528336 (+ 1271us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:38:39.528358 (+ 22us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:38:39.528653 (+ 295us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:38:39.529118 (+ 465us) client_negotiation.cc:769] Sending connection context
0809 15:38:39.529423 (+ 305us) client_negotiation.cc:240] Negotiation successful
0809 15:38:39.529504 (+ 81us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":849,"mutex_wait_us":506,"thread_start_us":82,"threads_started":1}
I20260809 15:38:39.530097 20483 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:38:39.523260 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33094 (local address 127.3.89.193:36955)
0809 15:38:39.523569 (+ 309us) server_negotiation.cc:206] Beginning negotiation
0809 15:38:39.523586 (+ 17us) server_negotiation.cc:399] Waiting for connection header
0809 15:38:39.523611 (+ 25us) server_negotiation.cc:407] Connection header received
0809 15:38:39.523691 (+ 80us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 15:38:39.523696 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 15:38:39.523895 (+ 199us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 15:38:39.524065 (+ 170us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 15:38:39.524258 (+ 193us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:38:39.525481 (+ 1223us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:38:39.527188 (+ 1707us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:38:39.528034 (+ 846us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:38:39.528142 (+ 108us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:38:39.528270 (+ 128us) server_negotiation.cc:1035] Waiting for connection context
0809 15:38:39.529839 (+ 1569us) server_negotiation.cc:299] Negotiation successful
0809 15:38:39.529952 (+ 113us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":213,"thread_start_us":113,"threads_started":1}
I20260809 15:38:39.531832 20393 tablet_service.cc:1467] Processing CreateTablet for tablet f8a4a8a731934dbf8c0716edb5004f83 (DEFAULT_TABLE table=test-table [id=4145f9c64db048f399b8c8e1804a7156]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 15:38:39.532935 20393 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f8a4a8a731934dbf8c0716edb5004f83. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 15:38:39.532570 20104 tablet_service.cc:1467] Processing CreateTablet for tablet f8a4a8a731934dbf8c0716edb5004f83 (DEFAULT_TABLE table=test-table [id=4145f9c64db048f399b8c8e1804a7156]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 15:38:39.532570 20249 tablet_service.cc:1467] Processing CreateTablet for tablet f8a4a8a731934dbf8c0716edb5004f83 (DEFAULT_TABLE table=test-table [id=4145f9c64db048f399b8c8e1804a7156]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 15:38:39.533587 20249 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f8a4a8a731934dbf8c0716edb5004f83. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 15:38:39.533587 20104 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f8a4a8a731934dbf8c0716edb5004f83. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 15:38:39.539788 20484 tablet_bootstrap.cc:492] T f8a4a8a731934dbf8c0716edb5004f83 P 440d47534a314ca88247d80185c766f8: Bootstrap starting.
I20260809 15:38:39.541061 20486 tablet_bootstrap.cc:492] T f8a4a8a731934dbf8c0716edb5004f83 P 7ba1b776d4734e909008ca7c997fde5f: Bootstrap starting.
I20260809 15:38:39.541226 20485 tablet_bootstrap.cc:492] T f8a4a8a731934dbf8c0716edb5004f83 P 23bd656811574169affd015915dd21e5: Bootstrap starting.
I20260809 15:38:39.542416 20484 tablet_bootstrap.cc:654] T f8a4a8a731934dbf8c0716edb5004f83 P 440d47534a314ca88247d80185c766f8: Neither blocks nor log segments found. Creating new log.
I20260809 15:38:39.543243 20485 tablet_bootstrap.cc:654] T f8a4a8a731934dbf8c0716edb5004f83 P 23bd656811574169affd015915dd21e5: Neither blocks nor log segments found. Creating new log.
I20260809 15:38:39.543267 20484 log.cc:824] T f8a4a8a731934dbf8c0716edb5004f83 P 440d47534a314ca88247d80185c766f8: Log is configured to *not* fsync() on all Append() calls
I20260809 15:38:39.543627 20486 tablet_bootstrap.cc:654] T f8a4a8a731934dbf8c0716edb5004f83 P 7ba1b776d4734e909008ca7c997fde5f: Neither blocks nor log segments found. Creating new log.
I20260809 15:38:39.544123 20485 log.cc:824] T f8a4a8a731934dbf8c0716edb5004f83 P 23bd656811574169affd015915dd21e5: Log is configured to *not* fsync() on all Append() calls
I20260809 15:38:39.544466 20486 log.cc:824] T f8a4a8a731934dbf8c0716edb5004f83 P 7ba1b776d4734e909008ca7c997fde5f: Log is configured to *not* fsync() on all Append() calls
I20260809 15:38:39.545751 20484 tablet_bootstrap.cc:492] T f8a4a8a731934dbf8c0716edb5004f83 P 440d47534a314ca88247d80185c766f8: No bootstrap required, opened a new log
I20260809 15:38:39.545967 20484 ts_tablet_manager.cc:1397] T f8a4a8a731934dbf8c0716edb5004f83 P 440d47534a314ca88247d80185c766f8: Time spent bootstrapping tablet: real 0.006s user 0.004s sys 0.000s
I20260809 15:38:39.546160 20485 tablet_bootstrap.cc:492] T f8a4a8a731934dbf8c0716edb5004f83 P 23bd656811574169affd015915dd21e5: No bootstrap required, opened a new log
I20260809 15:38:39.546208 20486 tablet_bootstrap.cc:492] T f8a4a8a731934dbf8c0716edb5004f83 P 7ba1b776d4734e909008ca7c997fde5f: No bootstrap required, opened a new log
I20260809 15:38:39.546346 20486 ts_tablet_manager.cc:1397] T f8a4a8a731934dbf8c0716edb5004f83 P 7ba1b776d4734e909008ca7c997fde5f: Time spent bootstrapping tablet: real 0.005s user 0.004s sys 0.000s
I20260809 15:38:39.546347 20485 ts_tablet_manager.cc:1397] T f8a4a8a731934dbf8c0716edb5004f83 P 23bd656811574169affd015915dd21e5: Time spent bootstrapping tablet: real 0.005s user 0.004s sys 0.000s
I20260809 15:38:39.549119 20484 raft_consensus.cc:348] T f8a4a8a731934dbf8c0716edb5004f83 P 440d47534a314ca88247d80185c766f8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "440d47534a314ca88247d80185c766f8" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 44003 } } peers { permanent_uuid: "23bd656811574169affd015915dd21e5" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 36955 } } peers { permanent_uuid: "7ba1b776d4734e909008ca7c997fde5f" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 32773 } }
I20260809 15:38:39.549384 20484 raft_consensus.cc:374] T f8a4a8a731934dbf8c0716edb5004f83 P 440d47534a314ca88247d80185c766f8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 15:38:39.549472 20484 raft_consensus.cc:729] T f8a4a8a731934dbf8c0716edb5004f83 P 440d47534a314ca88247d80185c766f8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 440d47534a314ca88247d80185c766f8, State: Initialized, Role: FOLLOWER
I20260809 15:38:39.549939 20484 consensus_queue.cc:260] T f8a4a8a731934dbf8c0716edb5004f83 P 440d47534a314ca88247d80185c766f8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "440d47534a314ca88247d80185c766f8" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 44003 } } peers { permanent_uuid: "23bd656811574169affd015915dd21e5" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 36955 } } peers { permanent_uuid: "7ba1b776d4734e909008ca7c997fde5f" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 32773 } }
I20260809 15:38:39.550199 20486 raft_consensus.cc:348] T f8a4a8a731934dbf8c0716edb5004f83 P 7ba1b776d4734e909008ca7c997fde5f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "440d47534a314ca88247d80185c766f8" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 44003 } } peers { permanent_uuid: "23bd656811574169affd015915dd21e5" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 36955 } } peers { permanent_uuid: "7ba1b776d4734e909008ca7c997fde5f" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 32773 } }
I20260809 15:38:39.550523 20486 raft_consensus.cc:374] T f8a4a8a731934dbf8c0716edb5004f83 P 7ba1b776d4734e909008ca7c997fde5f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 15:38:39.550626 20486 raft_consensus.cc:729] T f8a4a8a731934dbf8c0716edb5004f83 P 7ba1b776d4734e909008ca7c997fde5f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7ba1b776d4734e909008ca7c997fde5f, State: Initialized, Role: FOLLOWER
I20260809 15:38:39.550668 20485 raft_consensus.cc:348] T f8a4a8a731934dbf8c0716edb5004f83 P 23bd656811574169affd015915dd21e5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "440d47534a314ca88247d80185c766f8" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 44003 } } peers { permanent_uuid: "23bd656811574169affd015915dd21e5" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 36955 } } peers { permanent_uuid: "7ba1b776d4734e909008ca7c997fde5f" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 32773 } }
I20260809 15:38:39.550936 20485 raft_consensus.cc:374] T f8a4a8a731934dbf8c0716edb5004f83 P 23bd656811574169affd015915dd21e5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 15:38:39.551056 20485 raft_consensus.cc:729] T f8a4a8a731934dbf8c0716edb5004f83 P 23bd656811574169affd015915dd21e5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 23bd656811574169affd015915dd21e5, State: Initialized, Role: FOLLOWER
I20260809 15:38:39.551120 20484 ts_tablet_manager.cc:1428] T f8a4a8a731934dbf8c0716edb5004f83 P 440d47534a314ca88247d80185c766f8: Time spent starting tablet: real 0.005s user 0.007s sys 0.000s
I20260809 15:38:39.551208 20486 consensus_queue.cc:260] T f8a4a8a731934dbf8c0716edb5004f83 P 7ba1b776d4734e909008ca7c997fde5f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "440d47534a314ca88247d80185c766f8" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 44003 } } peers { permanent_uuid: "23bd656811574169affd015915dd21e5" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 36955 } } peers { permanent_uuid: "7ba1b776d4734e909008ca7c997fde5f" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 32773 } }
I20260809 15:38:39.551446 20467 heartbeater.cc:503] Master 127.3.89.254:41873 was elected leader, sending a full tablet report...
I20260809 15:38:39.551481 20485 consensus_queue.cc:260] T f8a4a8a731934dbf8c0716edb5004f83 P 23bd656811574169affd015915dd21e5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "440d47534a314ca88247d80185c766f8" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 44003 } } peers { permanent_uuid: "23bd656811574169affd015915dd21e5" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 36955 } } peers { permanent_uuid: "7ba1b776d4734e909008ca7c997fde5f" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 32773 } }
I20260809 15:38:39.552048 20322 heartbeater.cc:503] Master 127.3.89.254:41873 was elected leader, sending a full tablet report...
I20260809 15:38:39.552274 20486 ts_tablet_manager.cc:1428] T f8a4a8a731934dbf8c0716edb5004f83 P 7ba1b776d4734e909008ca7c997fde5f: Time spent starting tablet: real 0.006s user 0.006s sys 0.000s
I20260809 15:38:39.552376 20485 ts_tablet_manager.cc:1428] T f8a4a8a731934dbf8c0716edb5004f83 P 23bd656811574169affd015915dd21e5: Time spent starting tablet: real 0.006s user 0.003s sys 0.003s
I20260809 15:38:39.552655 20179 heartbeater.cc:503] Master 127.3.89.254:41873 was elected leader, sending a full tablet report...
I20260809 15:38:39.564687 20493 raft_consensus.cc:482] T f8a4a8a731934dbf8c0716edb5004f83 P 23bd656811574169affd015915dd21e5 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 15:38:39.564849 20493 raft_consensus.cc:504] T f8a4a8a731934dbf8c0716edb5004f83 P 23bd656811574169affd015915dd21e5 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "440d47534a314ca88247d80185c766f8" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 44003 } } peers { permanent_uuid: "23bd656811574169affd015915dd21e5" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 36955 } } peers { permanent_uuid: "7ba1b776d4734e909008ca7c997fde5f" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 32773 } }
I20260809 15:38:39.565927 20493 leader_election.cc:288] T f8a4a8a731934dbf8c0716edb5004f83 P 23bd656811574169affd015915dd21e5 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 440d47534a314ca88247d80185c766f8 (127.3.89.195:44003), 7ba1b776d4734e909008ca7c997fde5f (127.3.89.194:32773)
I20260809 15:38:39.571363 20480 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:38:39.566191 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.89.193:55619 (local address 127.3.89.195:44003)
0809 15:38:39.566334 (+ 143us) server_negotiation.cc:206] Beginning negotiation
0809 15:38:39.566339 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 15:38:39.566652 (+ 313us) server_negotiation.cc:407] Connection header received
0809 15:38:39.566823 (+ 171us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 15:38:39.566827 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 15:38:39.566882 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 15:38:39.567005 (+ 123us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 15:38:39.567438 (+ 433us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:38:39.568233 (+ 795us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:38:39.570089 (+ 1856us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:38:39.570704 (+ 615us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:38:39.570824 (+ 120us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:38:39.570873 (+ 49us) server_negotiation.cc:1035] Waiting for connection context
0809 15:38:39.571172 (+ 299us) server_negotiation.cc:299] Negotiation successful
0809 15:38:39.571238 (+ 66us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":68}
I20260809 15:38:39.571363 20482 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:38:39.566288 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.89.193:54571 (local address 127.3.89.194:32773)
0809 15:38:39.566411 (+ 123us) server_negotiation.cc:206] Beginning negotiation
0809 15:38:39.566416 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 15:38:39.566706 (+ 290us) server_negotiation.cc:407] Connection header received
0809 15:38:39.566832 (+ 126us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 15:38:39.566835 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 15:38:39.566882 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 15:38:39.567004 (+ 122us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 15:38:39.567438 (+ 434us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:38:39.568252 (+ 814us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:38:39.570089 (+ 1837us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:38:39.570705 (+ 616us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:38:39.570826 (+ 121us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:38:39.570873 (+ 47us) server_negotiation.cc:1035] Waiting for connection context
0809 15:38:39.571172 (+ 299us) server_negotiation.cc:299] Negotiation successful
0809 15:38:39.571238 (+ 66us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":52}
I20260809 15:38:39.571527 20495 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:38:39.566170 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.89.194:32773 (local address 127.3.89.193:54571)
0809 15:38:39.566636 (+ 466us) negotiation.cc:107] Waiting for socket to connect
0809 15:38:39.566648 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0809 15:38:39.566731 (+ 83us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 15:38:39.567017 (+ 286us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 15:38:39.567020 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 15:38:39.567061 (+ 41us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 15:38:39.567325 (+ 264us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:38:39.567332 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:38:39.568362 (+ 1030us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:38:39.568365 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:38:39.569964 (+ 1599us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:38:39.569975 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:38:39.570834 (+ 859us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:38:39.570838 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:38:39.571015 (+ 177us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:38:39.571075 (+ 60us) client_negotiation.cc:769] Sending connection context
0809 15:38:39.571152 (+ 77us) client_negotiation.cc:240] Negotiation successful
0809 15:38:39.571325 (+ 173us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":402,"mutex_wait_us":237,"spinlock_wait_cycles":4352,"thread_start_us":121,"threads_started":1}
I20260809 15:38:39.571575 20494 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:38:39.566111 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.89.195:44003 (local address 127.3.89.193:55619)
0809 15:38:39.566534 (+ 423us) negotiation.cc:107] Waiting for socket to connect
0809 15:38:39.566552 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0809 15:38:39.566676 (+ 124us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 15:38:39.567017 (+ 341us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 15:38:39.567020 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 15:38:39.567060 (+ 40us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 15:38:39.567325 (+ 265us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:38:39.567332 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:38:39.568361 (+ 1029us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:38:39.568365 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:38:39.569965 (+ 1600us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:38:39.569975 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:38:39.570834 (+ 859us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:38:39.570838 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:38:39.571026 (+ 188us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:38:39.571072 (+ 46us) client_negotiation.cc:769] Sending connection context
0809 15:38:39.571152 (+ 80us) client_negotiation.cc:240] Negotiation successful
0809 15:38:39.571325 (+ 173us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":322,"mutex_wait_us":319,"spinlock_wait_cycles":37248,"thread_start_us":125,"threads_started":1}
I20260809 15:38:39.572151 20270 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f8a4a8a731934dbf8c0716edb5004f83" candidate_uuid: "23bd656811574169affd015915dd21e5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7ba1b776d4734e909008ca7c997fde5f" is_pre_election: true
I20260809 15:38:39.572165 20414 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f8a4a8a731934dbf8c0716edb5004f83" candidate_uuid: "23bd656811574169affd015915dd21e5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "440d47534a314ca88247d80185c766f8" is_pre_election: true
I20260809 15:38:39.572431 20414 raft_consensus.cc:2454] T f8a4a8a731934dbf8c0716edb5004f83 P 440d47534a314ca88247d80185c766f8 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 23bd656811574169affd015915dd21e5 in term 0.
I20260809 15:38:39.572434 20270 raft_consensus.cc:2454] T f8a4a8a731934dbf8c0716edb5004f83 P 7ba1b776d4734e909008ca7c997fde5f [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 23bd656811574169affd015915dd21e5 in term 0.
I20260809 15:38:39.572907 20057 leader_election.cc:302] T f8a4a8a731934dbf8c0716edb5004f83 P 23bd656811574169affd015915dd21e5 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 23bd656811574169affd015915dd21e5, 7ba1b776d4734e909008ca7c997fde5f; no voters:
I20260809 15:38:39.573189 20493 raft_consensus.cc:2781] T f8a4a8a731934dbf8c0716edb5004f83 P 23bd656811574169affd015915dd21e5 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260809 15:38:39.573267 20493 raft_consensus.cc:482] T f8a4a8a731934dbf8c0716edb5004f83 P 23bd656811574169affd015915dd21e5 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 15:38:39.573298 20493 raft_consensus.cc:3037] T f8a4a8a731934dbf8c0716edb5004f83 P 23bd656811574169affd015915dd21e5 [term 0 FOLLOWER]: Advancing to term 1
I20260809 15:38:39.574331 20493 raft_consensus.cc:504] T f8a4a8a731934dbf8c0716edb5004f83 P 23bd656811574169affd015915dd21e5 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "440d47534a314ca88247d80185c766f8" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 44003 } } peers { permanent_uuid: "23bd656811574169affd015915dd21e5" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 36955 } } peers { permanent_uuid: "7ba1b776d4734e909008ca7c997fde5f" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 32773 } }
I20260809 15:38:39.574669 20493 leader_election.cc:288] T f8a4a8a731934dbf8c0716edb5004f83 P 23bd656811574169affd015915dd21e5 [CANDIDATE]: Term 1 election: Requested vote from peers 440d47534a314ca88247d80185c766f8 (127.3.89.195:44003), 7ba1b776d4734e909008ca7c997fde5f (127.3.89.194:32773)
I20260809 15:38:39.575146 20414 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f8a4a8a731934dbf8c0716edb5004f83" candidate_uuid: "23bd656811574169affd015915dd21e5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "440d47534a314ca88247d80185c766f8"
I20260809 15:38:39.575119 20270 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f8a4a8a731934dbf8c0716edb5004f83" candidate_uuid: "23bd656811574169affd015915dd21e5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7ba1b776d4734e909008ca7c997fde5f"
I20260809 15:38:39.575277 20414 raft_consensus.cc:3037] T f8a4a8a731934dbf8c0716edb5004f83 P 440d47534a314ca88247d80185c766f8 [term 0 FOLLOWER]: Advancing to term 1
I20260809 15:38:39.575271 20270 raft_consensus.cc:3037] T f8a4a8a731934dbf8c0716edb5004f83 P 7ba1b776d4734e909008ca7c997fde5f [term 0 FOLLOWER]: Advancing to term 1
I20260809 15:38:39.576649 20414 raft_consensus.cc:2454] T f8a4a8a731934dbf8c0716edb5004f83 P 440d47534a314ca88247d80185c766f8 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 23bd656811574169affd015915dd21e5 in term 1.
I20260809 15:38:39.576649 20270 raft_consensus.cc:2454] T f8a4a8a731934dbf8c0716edb5004f83 P 7ba1b776d4734e909008ca7c997fde5f [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 23bd656811574169affd015915dd21e5 in term 1.
I20260809 15:38:39.576987 20055 leader_election.cc:302] T f8a4a8a731934dbf8c0716edb5004f83 P 23bd656811574169affd015915dd21e5 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 23bd656811574169affd015915dd21e5, 440d47534a314ca88247d80185c766f8; no voters:
I20260809 15:38:39.577190 20493 raft_consensus.cc:2781] T f8a4a8a731934dbf8c0716edb5004f83 P 23bd656811574169affd015915dd21e5 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 15:38:39.577421 20493 raft_consensus.cc:686] T f8a4a8a731934dbf8c0716edb5004f83 P 23bd656811574169affd015915dd21e5 [term 1 LEADER]: Becoming Leader. State: Replica: 23bd656811574169affd015915dd21e5, State: Running, Role: LEADER
I20260809 15:38:39.577750 20493 consensus_queue.cc:237] T f8a4a8a731934dbf8c0716edb5004f83 P 23bd656811574169affd015915dd21e5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "440d47534a314ca88247d80185c766f8" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 44003 } } peers { permanent_uuid: "23bd656811574169affd015915dd21e5" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 36955 } } peers { permanent_uuid: "7ba1b776d4734e909008ca7c997fde5f" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 32773 } }
I20260809 15:38:39.580706 19968 catalog_manager.cc:5495] T f8a4a8a731934dbf8c0716edb5004f83 P 23bd656811574169affd015915dd21e5 reported cstate change: term changed from 0 to 1, leader changed from <none> to 23bd656811574169affd015915dd21e5 (127.3.89.193). New cstate: current_term: 1 leader_uuid: "23bd656811574169affd015915dd21e5" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "440d47534a314ca88247d80185c766f8" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 44003 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "23bd656811574169affd015915dd21e5" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 36955 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "7ba1b776d4734e909008ca7c997fde5f" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 32773 } health_report { overall_health: UNKNOWN } } }
I20260809 15:38:39.605789 20483 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:38:39.602181 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33100 (local address 127.3.89.193:36955)
0809 15:38:39.602344 (+ 163us) server_negotiation.cc:206] Beginning negotiation
0809 15:38:39.602349 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 15:38:39.602378 (+ 29us) server_negotiation.cc:407] Connection header received
0809 15:38:39.602462 (+ 84us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 15:38:39.602464 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 15:38:39.602516 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 15:38:39.602623 (+ 107us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 15:38:39.602924 (+ 301us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:38:39.603476 (+ 552us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:38:39.604340 (+ 864us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:38:39.604562 (+ 222us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:38:39.604672 (+ 110us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:38:39.604980 (+ 308us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 15:38:39.605390 (+ 410us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 15:38:39.605520 (+ 130us) server_negotiation.cc:1035] Waiting for connection context
0809 15:38:39.605585 (+ 65us) server_negotiation.cc:299] Negotiation successful
0809 15:38:39.605641 (+ 56us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":68}
I20260809 15:38:39.611600 20414 raft_consensus.cc:1261] T f8a4a8a731934dbf8c0716edb5004f83 P 440d47534a314ca88247d80185c766f8 [term 1 FOLLOWER]: Refusing update from remote peer 23bd656811574169affd015915dd21e5: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 15:38:39.611616 20270 raft_consensus.cc:1261] T f8a4a8a731934dbf8c0716edb5004f83 P 7ba1b776d4734e909008ca7c997fde5f [term 1 FOLLOWER]: Refusing update from remote peer 23bd656811574169affd015915dd21e5: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 15:38:39.612334 20496 consensus_queue.cc:1035] T f8a4a8a731934dbf8c0716edb5004f83 P 23bd656811574169affd015915dd21e5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7ba1b776d4734e909008ca7c997fde5f" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 32773 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 15:38:39.612517 20493 consensus_queue.cc:1035] T f8a4a8a731934dbf8c0716edb5004f83 P 23bd656811574169affd015915dd21e5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "440d47534a314ca88247d80185c766f8" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 44003 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 15:38:39.620718 20500 mvcc.cc:205] Tried to move back new op lower bound from 7316643510721740800 to 7316643510592929792. Current Snapshot: MvccSnapshot[applied={T|T < 7316643510721740800}]
I20260809 15:38:39.628831 19967 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:54754:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 15:38:39.629007 19967 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:54754:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 15:38:39.631382 19967 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 210f7703a6ee4761acec0985806eb015: Sending DeleteTablet for 3 replicas of tablet f8a4a8a731934dbf8c0716edb5004f83
I20260809 15:38:39.632119 20393 tablet_service.cc:1514] Processing DeleteTablet for tablet f8a4a8a731934dbf8c0716edb5004f83 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 15:38:39 UTC) from {username='kudu', principal='kudu/127.3.89.254@KRBTEST.COM'} at 127.0.0.1:45998
I20260809 15:38:39.632213 20104 tablet_service.cc:1514] Processing DeleteTablet for tablet f8a4a8a731934dbf8c0716edb5004f83 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 15:38:39 UTC) from {username='kudu', principal='kudu/127.3.89.254@KRBTEST.COM'} at 127.0.0.1:33094
I20260809 15:38:39.632596 20510 tablet_replica.cc:330] T f8a4a8a731934dbf8c0716edb5004f83 P 23bd656811574169affd015915dd21e5: stopping tablet replica
I20260809 15:38:39.632597 20509 tablet_replica.cc:330] T f8a4a8a731934dbf8c0716edb5004f83 P 440d47534a314ca88247d80185c766f8: stopping tablet replica
I20260809 15:38:39.632575 20249 tablet_service.cc:1514] Processing DeleteTablet for tablet f8a4a8a731934dbf8c0716edb5004f83 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 15:38:39 UTC) from {username='kudu', principal='kudu/127.3.89.254@KRBTEST.COM'} at 127.0.0.1:34344
I20260809 15:38:39.632925 20510 raft_consensus.cc:2229] T f8a4a8a731934dbf8c0716edb5004f83 P 23bd656811574169affd015915dd21e5 [term 1 LEADER]: Raft consensus shutting down.
I20260809 15:38:39.632898 20509 raft_consensus.cc:2229] T f8a4a8a731934dbf8c0716edb5004f83 P 440d47534a314ca88247d80185c766f8 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 15:38:39.633065 3431 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskm314YR/build/debug/bin/kudu with pid 20038
I20260809 15:38:39.634564 20511 tablet_replica.cc:330] T f8a4a8a731934dbf8c0716edb5004f83 P 7ba1b776d4734e909008ca7c997fde5f: stopping tablet replica
I20260809 15:38:39.634884 20511 raft_consensus.cc:2229] T f8a4a8a731934dbf8c0716edb5004f83 P 7ba1b776d4734e909008ca7c997fde5f [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 15:38:39.635192 20511 raft_consensus.cc:2258] T f8a4a8a731934dbf8c0716edb5004f83 P 7ba1b776d4734e909008ca7c997fde5f [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 15:38:39.635432 20509 raft_consensus.cc:2258] T f8a4a8a731934dbf8c0716edb5004f83 P 440d47534a314ca88247d80185c766f8 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 15:38:39.637012 20509 ts_tablet_manager.cc:1905] T f8a4a8a731934dbf8c0716edb5004f83 P 440d47534a314ca88247d80185c766f8: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 15:38:39.637634 20511 ts_tablet_manager.cc:1905] T f8a4a8a731934dbf8c0716edb5004f83 P 7ba1b776d4734e909008ca7c997fde5f: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 15:38:39.639995 20509 ts_tablet_manager.cc:1918] T f8a4a8a731934dbf8c0716edb5004f83 P 440d47534a314ca88247d80185c766f8: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 15:38:39.640102 20509 log.cc:1196] T f8a4a8a731934dbf8c0716edb5004f83 P 440d47534a314ca88247d80185c766f8: Deleting WAL directory at /tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786289732823074-3431-0/minicluster-data/ts-2/wal/wals/f8a4a8a731934dbf8c0716edb5004f83
I20260809 15:38:39.640406 20511 ts_tablet_manager.cc:1918] T f8a4a8a731934dbf8c0716edb5004f83 P 7ba1b776d4734e909008ca7c997fde5f: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 15:38:39.640465 20509 ts_tablet_manager.cc:1939] T f8a4a8a731934dbf8c0716edb5004f83 P 440d47534a314ca88247d80185c766f8: Deleting consensus metadata
I20260809 15:38:39.640518 20511 log.cc:1196] T f8a4a8a731934dbf8c0716edb5004f83 P 7ba1b776d4734e909008ca7c997fde5f: Deleting WAL directory at /tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786289732823074-3431-0/minicluster-data/ts-1/wal/wals/f8a4a8a731934dbf8c0716edb5004f83
I20260809 15:38:39.640859 20511 ts_tablet_manager.cc:1939] T f8a4a8a731934dbf8c0716edb5004f83 P 7ba1b776d4734e909008ca7c997fde5f: Deleting consensus metadata
I20260809 15:38:39.641665 19952 catalog_manager.cc:4841] TS 440d47534a314ca88247d80185c766f8 (127.3.89.195:44003): tablet f8a4a8a731934dbf8c0716edb5004f83 (table test-table [id=4145f9c64db048f399b8c8e1804a7156]) successfully deleted
I20260809 15:38:39.641856 19953 catalog_manager.cc:4841] TS 7ba1b776d4734e909008ca7c997fde5f (127.3.89.194:32773): tablet f8a4a8a731934dbf8c0716edb5004f83 (table test-table [id=4145f9c64db048f399b8c8e1804a7156]) successfully deleted
W20260809 15:38:39.642149 19953 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.3.89.193:36955 (error 108)
I20260809 15:38:39.642380 3431 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskm314YR/build/debug/bin/kudu with pid 20184
I20260809 15:38:39.645443 20481 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:38:39.645088 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.89.193:36955 (local address 127.0.0.1:33102)
0809 15:38:39.645265 (+ 177us) negotiation.cc:107] Waiting for socket to connect
0809 15:38:39.645347 (+ 82us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.89.193:36955: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":54}
W20260809 15:38:39.645632 19953 catalog_manager.cc:4584] TS 23bd656811574169affd015915dd21e5 (127.3.89.193:36955): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet f8a4a8a731934dbf8c0716edb5004f83: Network error: Client connection negotiation failed: client connection to 127.3.89.193:36955: connect: Connection refused (error 111)
I20260809 15:38:39.650308 3431 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskm314YR/build/debug/bin/kudu with pid 20327
I20260809 15:38:39.658247 3431 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskm314YR/build/debug/bin/kudu with pid 19934
I20260809 15:38:59.666872 3431 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskm314YR/build/debug/bin/kudu
/tmp/dist-test-taskm314YR/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786289732823074-3431-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786289732823074-3431-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786289732823074-3431-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786289732823074-3431-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.3.89.254:41873
--webserver_interface=127.3.89.254
--webserver_port=35627
--keytab_file=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786289732823074-3431-0/krb5kdc/kudu_127.3.89.254.keytab
--principal=kudu/127.3.89.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.3.89.212:37503
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.3.89.254:41873
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20
--tsk_private_key_password_cmd=echo foo with env {KRB5CCNAME=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786289732823074-3431-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786289732823074-3431-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786289732823074-3431-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 15:38:59.778332 20513 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 15:38:59.778585 20513 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 15:38:59.778625 20513 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 15:38:59.782655 20513 flags.cc:425] Enabled experimental flag: --authn_token_validity_seconds=20
W20260809 15:38:59.782703 20513 flags.cc:425] Enabled experimental flag: --authz_token_validity_seconds=20
W20260809 15:38:59.782718 20513 flags.cc:425] Enabled experimental flag: --tsk_rotation_seconds=20
W20260809 15:38:59.782732 20513 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 15:38:59.782748 20513 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 15:38:59.782766 20513 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 15:38:59.782785 20513 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 15:38:59.782800 20513 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 15:38:59.787945 20513 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.3.89.212:37503
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786289732823074-3431-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786289732823074-3431-0/minicluster-data/master-0/wal
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_private_key_password_cmd=echo foo
--tsk_rotation_seconds=20
--ipki_ca_key_size=768
--master_addresses=127.3.89.254:41873
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786289732823074-3431-0/krb5kdc/kudu_127.3.89.254.keytab
--principal=kudu/127.3.89.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.3.89.254:41873
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786289732823074-3431-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.3.89.254
--webserver_port=35627
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.20513
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786289732823074-3431-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 09 Aug 2026 14:46:55 UTC on ccycloud.quasar-yswkmf.root.comops.site
build id 4576540
I20260809 15:38:59.788945 20513 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 15:38:59.789939 20513 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260809 15:38:59.803545 20513 server_base.cc:1034] running on GCE node
W20260809 15:38:59.803501 20521 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 15:38:59.803503 20519 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 15:38:59.803521 20523 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 15:38:59.804427 20513 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 15:38:59.805555 20513 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 15:38:59.806746 20513 hybrid_clock.cc:648] HybridClock initialized: now 1786289939806715 us; error 43 us; skew 500 ppm
Aug 09 15:38:59 dist-test-slave-2w3w krb5kdc[19918](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786289939, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.89.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 15:38:59.812937 20513 init.cc:377] Logged in from keytab as kudu/127.3.89.254@KRBTEST.COM (short username kudu)
I20260809 15:38:59.814046 20513 webserver.cc:460] Webserver started at http://127.3.89.254:35627/ using document root <none> and password file <none>
I20260809 15:38:59.814627 20513 fs_manager.cc:359] Metadata directory not provided
I20260809 15:38:59.814756 20513 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 15:38:59.818593 20513 fs_manager.cc:709] Time spent opening directory manager: real 0.002s user 0.003s sys 0.000s
I20260809 15:38:59.820766 20530 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 15:38:59.821913 20513 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.000s sys 0.002s
I20260809 15:38:59.822054 20513 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786289732823074-3431-0/minicluster-data/master-0/data,/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786289732823074-3431-0/minicluster-data/master-0/wal
uuid: "210f7703a6ee4761acec0985806eb015"
format_stamp: "Formatted at 2026-08-09 15:38:38 on dist-test-slave-2w3w"
server_key: "03ecf0ac536692172efc79bd865fd7d4"
server_key_iv: "397c712c73463366eb9a3a049bad8e83"
server_key_version: "encryptionkey@0"
I20260809 15:38:59.822654 20513 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786289732823074-3431-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786289732823074-3431-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786289732823074-3431-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 15:38:59.844714 20513 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 15:38:59.847455 20513 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 15:38:59.847661 20513 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 15:38:59.854947 20513 rpc_server.cc:307] RPC server started. Bound to: 127.3.89.254:41873
I20260809 15:38:59.855293 20591 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.3.89.254:41873 every 8 connection(s)
I20260809 15:38:59.856063 20513 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786289732823074-3431-0/minicluster-data/master-0/data/info.pb
I20260809 15:38:59.860424 20592 sys_catalog.cc:263] Verifying existing consensus state
I20260809 15:38:59.861994 20592 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 210f7703a6ee4761acec0985806eb015: Bootstrap starting.
I20260809 15:38:59.864629 3431 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskm314YR/build/debug/bin/kudu as pid 20513
I20260809 15:38:59.865123 3431 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskm314YR/build/debug/bin/kudu
/tmp/dist-test-taskm314YR/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786289732823074-3431-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786289732823074-3431-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786289732823074-3431-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786289732823074-3431-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.3.89.193:36955
--local_ip_for_outbound_sockets=127.3.89.193
--tserver_master_addrs=127.3.89.254:41873
--webserver_port=44001
--webserver_interface=127.3.89.193
--keytab_file=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786289732823074-3431-0/krb5kdc/kudu_127.3.89.193.keytab
--principal=kudu/127.3.89.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.3.89.212:37503
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786289732823074-3431-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786289732823074-3431-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786289732823074-3431-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260809 15:38:59.880281 20592 log.cc:824] T 00000000000000000000000000000000 P 210f7703a6ee4761acec0985806eb015: Log is configured to *not* fsync() on all Append() calls
I20260809 15:38:59.888928 20592 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 210f7703a6ee4761acec0985806eb015: Bootstrap replayed 1/1 log segments. Stats: ops{read=8 overwritten=0 applied=8 ignored=0} inserts{seen=5 ignored=0} mutations{seen=4 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260809 15:38:59.889329 20592 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 210f7703a6ee4761acec0985806eb015: Bootstrap complete.
I20260809 15:38:59.893488 20592 raft_consensus.cc:348] T 00000000000000000000000000000000 P 210f7703a6ee4761acec0985806eb015 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "210f7703a6ee4761acec0985806eb015" member_type: VOTER last_known_addr { host: "127.3.89.254" port: 41873 } }
I20260809 15:38:59.893857 20592 raft_consensus.cc:729] T 00000000000000000000000000000000 P 210f7703a6ee4761acec0985806eb015 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 210f7703a6ee4761acec0985806eb015, State: Initialized, Role: FOLLOWER
I20260809 15:38:59.894344 20592 consensus_queue.cc:260] T 00000000000000000000000000000000 P 210f7703a6ee4761acec0985806eb015 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 8, Last appended: 1.8, Last appended by leader: 8, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "210f7703a6ee4761acec0985806eb015" member_type: VOTER last_known_addr { host: "127.3.89.254" port: 41873 } }
I20260809 15:38:59.894459 20592 raft_consensus.cc:388] T 00000000000000000000000000000000 P 210f7703a6ee4761acec0985806eb015 [term 1 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 15:38:59.894556 20592 raft_consensus.cc:482] T 00000000000000000000000000000000 P 210f7703a6ee4761acec0985806eb015 [term 1 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 15:38:59.894667 20592 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 210f7703a6ee4761acec0985806eb015 [term 1 FOLLOWER]: Advancing to term 2
I20260809 15:38:59.896596 20592 raft_consensus.cc:504] T 00000000000000000000000000000000 P 210f7703a6ee4761acec0985806eb015 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "210f7703a6ee4761acec0985806eb015" member_type: VOTER last_known_addr { host: "127.3.89.254" port: 41873 } }
I20260809 15:38:59.896927 20592 leader_election.cc:302] T 00000000000000000000000000000000 P 210f7703a6ee4761acec0985806eb015 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 210f7703a6ee4761acec0985806eb015; no voters:
I20260809 15:38:59.897228 20592 leader_election.cc:288] T 00000000000000000000000000000000 P 210f7703a6ee4761acec0985806eb015 [CANDIDATE]: Term 2 election: Requested vote from peers
I20260809 15:38:59.897379 20596 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 210f7703a6ee4761acec0985806eb015 [term 2 FOLLOWER]: Leader election won for term 2
I20260809 15:38:59.897629 20596 raft_consensus.cc:686] T 00000000000000000000000000000000 P 210f7703a6ee4761acec0985806eb015 [term 2 LEADER]: Becoming Leader. State: Replica: 210f7703a6ee4761acec0985806eb015, State: Running, Role: LEADER
I20260809 15:38:59.897943 20596 consensus_queue.cc:237] T 00000000000000000000000000000000 P 210f7703a6ee4761acec0985806eb015 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 1.8, Last appended by leader: 8, Current term: 2, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "210f7703a6ee4761acec0985806eb015" member_type: VOTER last_known_addr { host: "127.3.89.254" port: 41873 } }
I20260809 15:38:59.898473 20592 sys_catalog.cc:564] T 00000000000000000000000000000000 P 210f7703a6ee4761acec0985806eb015 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 15:38:59.899327 20597 sys_catalog.cc:455] T 00000000000000000000000000000000 P 210f7703a6ee4761acec0985806eb015 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 2 leader_uuid: "210f7703a6ee4761acec0985806eb015" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "210f7703a6ee4761acec0985806eb015" member_type: VOTER last_known_addr { host: "127.3.89.254" port: 41873 } } }
I20260809 15:38:59.899477 20597 sys_catalog.cc:458] T 00000000000000000000000000000000 P 210f7703a6ee4761acec0985806eb015 [sys.catalog]: This master's current role is: LEADER
I20260809 15:38:59.900071 20598 sys_catalog.cc:455] T 00000000000000000000000000000000 P 210f7703a6ee4761acec0985806eb015 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 210f7703a6ee4761acec0985806eb015. Latest consensus state: current_term: 2 leader_uuid: "210f7703a6ee4761acec0985806eb015" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "210f7703a6ee4761acec0985806eb015" member_type: VOTER last_known_addr { host: "127.3.89.254" port: 41873 } } }
I20260809 15:38:59.900249 20598 sys_catalog.cc:458] T 00000000000000000000000000000000 P 210f7703a6ee4761acec0985806eb015 [sys.catalog]: This master's current role is: LEADER
I20260809 15:38:59.900437 20604 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 15:38:59.905102 20604 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 15:38:59.905697 20604 catalog_manager.cc:1259] Loaded cluster ID: bc02bf064fd44d40b13be3238ff5b7dc
I20260809 15:38:59.905755 20604 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 15:38:59.908245 20604 catalog_manager.cc:1504] Loading token signing keys...
I20260809 15:38:59.908924 20604 catalog_manager.cc:5879] T 00000000000000000000000000000000 P 210f7703a6ee4761acec0985806eb015: Loaded TSK: 0
I20260809 15:38:59.933944 20604 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 210f7703a6ee4761acec0985806eb015: Generated new TSK 1
I20260809 15:38:59.934199 20604 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260809 15:38:59.985925 20594 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 15:38:59.986179 20594 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 15:38:59.986220 20594 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 15:38:59.990197 20594 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 15:38:59.990257 20594 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 15:38:59.990314 20594 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.3.89.193
I20260809 15:38:59.995492 20594 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.3.89.212:37503
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786289732823074-3431-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786289732823074-3431-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786289732823074-3431-0/krb5kdc/kudu_127.3.89.193.keytab
--principal=kudu/127.3.89.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.3.89.193:36955
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786289732823074-3431-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.3.89.193
--webserver_port=44001
--tserver_master_addrs=127.3.89.254:41873
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.20594
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.3.89.193
--log_dir=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786289732823074-3431-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 09 Aug 2026 14:46:55 UTC on ccycloud.quasar-yswkmf.root.comops.site
build id 4576540
I20260809 15:38:59.996464 20594 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 15:38:59.997450 20594 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 15:39:00.007411 20620 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 15:39:00.007423 20623 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 15:39:00.007663 20621 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 15:39:00.007948 20594 server_base.cc:1034] running on GCE node
I20260809 15:39:00.008374 20594 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 15:39:00.009013 20594 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 15:39:00.010205 20594 hybrid_clock.cc:648] HybridClock initialized: now 1786289940010179 us; error 46 us; skew 500 ppm
Aug 09 15:39:00 dist-test-slave-2w3w krb5kdc[19918](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786289940, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.89.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 15:39:00.013316 20594 init.cc:377] Logged in from keytab as kudu/127.3.89.193@KRBTEST.COM (short username kudu)
I20260809 15:39:00.014544 20594 webserver.cc:460] Webserver started at http://127.3.89.193:44001/ using document root <none> and password file <none>
I20260809 15:39:00.015287 20594 fs_manager.cc:359] Metadata directory not provided
I20260809 15:39:00.015399 20594 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 15:39:00.020630 20594 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.004s sys 0.000s
I20260809 15:39:00.023343 20630 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 15:39:00.024789 20594 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260809 15:39:00.024904 20594 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786289732823074-3431-0/minicluster-data/ts-0/data,/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786289732823074-3431-0/minicluster-data/ts-0/wal
uuid: "23bd656811574169affd015915dd21e5"
format_stamp: "Formatted at 2026-08-09 15:38:38 on dist-test-slave-2w3w"
server_key: "a4aab3d472b4637e64f780d2befe66ff"
server_key_iv: "8968b27898f895919b075309b448b735"
server_key_version: "encryptionkey@0"
I20260809 15:39:00.025620 20594 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786289732823074-3431-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786289732823074-3431-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786289732823074-3431-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 15:39:00.040921 20594 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 15:39:00.044888 20594 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 15:39:00.045091 20594 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 15:39:00.045766 20594 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 15:39:00.047024 20637 ts_tablet_manager.cc:536] Loading tablet metadata (0/1 complete)
I20260809 15:39:00.051014 20594 ts_tablet_manager.cc:579] Loaded tablet metadata (1 total tablets, 1 live tablets)
I20260809 15:39:00.051088 20594 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.004s user 0.000s sys 0.000s
I20260809 15:39:00.051151 20594 ts_tablet_manager.cc:594] Registering tablets (0/1 complete)
I20260809 15:39:00.053145 20594 ts_tablet_manager.cc:610] Registered 1 tablets
I20260809 15:39:00.053201 20594 ts_tablet_manager.cc:589] Time spent register tablets: real 0.002s user 0.000s sys 0.000s
I20260809 15:39:00.053560 20637 tablet_bootstrap.cc:492] T f8a4a8a731934dbf8c0716edb5004f83 P 23bd656811574169affd015915dd21e5: Bootstrap starting.
I20260809 15:39:00.064081 20594 rpc_server.cc:307] RPC server started. Bound to: 127.3.89.193:36955
I20260809 15:39:00.064186 20755 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.3.89.193:36955 every 8 connection(s)
I20260809 15:39:00.065114 20594 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786289732823074-3431-0/minicluster-data/ts-0/data/info.pb
I20260809 15:39:00.072659 3431 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskm314YR/build/debug/bin/kudu as pid 20594
I20260809 15:39:00.073185 3431 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskm314YR/build/debug/bin/kudu
/tmp/dist-test-taskm314YR/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786289732823074-3431-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786289732823074-3431-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786289732823074-3431-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786289732823074-3431-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.3.89.194:32773
--local_ip_for_outbound_sockets=127.3.89.194
--tserver_master_addrs=127.3.89.254:41873
--webserver_port=39463
--webserver_interface=127.3.89.194
--keytab_file=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786289732823074-3431-0/krb5kdc/kudu_127.3.89.194.keytab
--principal=kudu/127.3.89.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.3.89.212:37503
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786289732823074-3431-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786289732823074-3431-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786289732823074-3431-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260809 15:39:00.073594 20637 log.cc:824] T f8a4a8a731934dbf8c0716edb5004f83 P 23bd656811574169affd015915dd21e5: Log is configured to *not* fsync() on all Append() calls
Aug 09 15:39:00 dist-test-slave-2w3w krb5kdc[19918](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786289940, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.89.193@KRBTEST.COM for kudu/127.3.89.254@KRBTEST.COM
I20260809 15:39:00.082521 20637 tablet_bootstrap.cc:492] T f8a4a8a731934dbf8c0716edb5004f83 P 23bd656811574169affd015915dd21e5: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=1 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260809 15:39:00.083001 20637 tablet_bootstrap.cc:492] T f8a4a8a731934dbf8c0716edb5004f83 P 23bd656811574169affd015915dd21e5: Bootstrap complete.
I20260809 15:39:00.084368 20637 ts_tablet_manager.cc:1397] T f8a4a8a731934dbf8c0716edb5004f83 P 23bd656811574169affd015915dd21e5: Time spent bootstrapping tablet: real 0.031s user 0.013s sys 0.016s
I20260809 15:39:00.084824 20761 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:39:00.067334 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.89.193:54213 (local address 127.3.89.254:41873)
0809 15:39:00.067754 (+ 420us) server_negotiation.cc:206] Beginning negotiation
0809 15:39:00.067777 (+ 23us) server_negotiation.cc:399] Waiting for connection header
0809 15:39:00.068112 (+ 335us) server_negotiation.cc:407] Connection header received
0809 15:39:00.069203 (+ 1091us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 15:39:00.069218 (+ 15us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 15:39:00.069666 (+ 448us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 15:39:00.070100 (+ 434us) server_negotiation.cc:226] Negotiated authn=SASL
0809 15:39:00.071175 (+ 1075us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:39:00.072751 (+ 1576us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:39:00.074488 (+ 1737us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:39:00.075027 (+ 539us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:39:00.075215 (+ 188us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:39:00.078559 (+ 3344us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 15:39:00.078566 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 15:39:00.078580 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 15:39:00.078612 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0809 15:39:00.080865 (+ 2253us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 15:39:00.081572 (+ 707us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 15:39:00.081577 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 15:39:00.081583 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0809 15:39:00.081735 (+ 152us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 15:39:00.082176 (+ 441us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 15:39:00.082180 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 15:39:00.082181 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 15:39:00.082653 (+ 472us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 15:39:00.082803 (+ 150us) server_negotiation.cc:1035] Waiting for connection context
0809 15:39:00.083537 (+ 734us) server_negotiation.cc:299] Negotiation successful
0809 15:39:00.083847 (+ 310us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":268,"thread_start_us":145,"threads_started":1}
I20260809 15:39:00.084836 20762 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:39:00.067268 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.89.254:41873 (local address 127.3.89.193:54213)
0809 15:39:00.067904 (+ 636us) negotiation.cc:107] Waiting for socket to connect
0809 15:39:00.067957 (+ 53us) client_negotiation.cc:174] Beginning negotiation
0809 15:39:00.068783 (+ 826us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 15:39:00.070134 (+ 1351us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 15:39:00.070140 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 15:39:00.070486 (+ 346us) client_negotiation.cc:189] Negotiated authn=SASL
0809 15:39:00.070921 (+ 435us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:39:00.070935 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:39:00.072893 (+ 1958us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:39:00.072897 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:39:00.074330 (+ 1433us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:39:00.074344 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:39:00.075346 (+ 1002us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:39:00.075350 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:39:00.075542 (+ 192us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:39:00.076258 (+ 716us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 15:39:00.076278 (+ 20us) client_negotiation.cc:656] calling sasl_client_start()
0809 15:39:00.078329 (+ 2051us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 15:39:00.081005 (+ 2676us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 15:39:00.081013 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 15:39:00.081028 (+ 15us) client_negotiation.cc:760] calling sasl_client_step()
0809 15:39:00.081438 (+ 410us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 15:39:00.081856 (+ 418us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 15:39:00.081861 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 15:39:00.081864 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0809 15:39:00.082058 (+ 194us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 15:39:00.082792 (+ 734us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 15:39:00.082799 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 15:39:00.083148 (+ 349us) client_negotiation.cc:769] Sending connection context
0809 15:39:00.083386 (+ 238us) client_negotiation.cc:240] Negotiation successful
0809 15:39:00.083752 (+ 366us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":429,"thread_start_us":161,"threads_started":1}
I20260809 15:39:00.087029 20637 raft_consensus.cc:348] T f8a4a8a731934dbf8c0716edb5004f83 P 23bd656811574169affd015915dd21e5 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "440d47534a314ca88247d80185c766f8" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 44003 } } peers { permanent_uuid: "23bd656811574169affd015915dd21e5" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 36955 } } peers { permanent_uuid: "7ba1b776d4734e909008ca7c997fde5f" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 32773 } }
I20260809 15:39:00.087577 20637 raft_consensus.cc:729] T f8a4a8a731934dbf8c0716edb5004f83 P 23bd656811574169affd015915dd21e5 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 23bd656811574169affd015915dd21e5, State: Initialized, Role: FOLLOWER
I20260809 15:39:00.087754 20757 heartbeater.cc:347] Connected to a master server at 127.3.89.254:41873
I20260809 15:39:00.087990 20757 heartbeater.cc:464] Registering TS with master...
I20260809 15:39:00.088034 20637 consensus_queue.cc:260] T f8a4a8a731934dbf8c0716edb5004f83 P 23bd656811574169affd015915dd21e5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "440d47534a314ca88247d80185c766f8" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 44003 } } peers { permanent_uuid: "23bd656811574169affd015915dd21e5" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 36955 } } peers { permanent_uuid: "7ba1b776d4734e909008ca7c997fde5f" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 32773 } }
I20260809 15:39:00.088511 20757 heartbeater.cc:511] Master 127.3.89.254:41873 requested a full tablet report, sending...
I20260809 15:39:00.088939 20637 ts_tablet_manager.cc:1428] T f8a4a8a731934dbf8c0716edb5004f83 P 23bd656811574169affd015915dd21e5: Time spent starting tablet: real 0.004s user 0.004s sys 0.001s
I20260809 15:39:00.090876 20549 ts_manager.cc:194] Registered new tserver with Master: 23bd656811574169affd015915dd21e5 (127.3.89.193:36955)
I20260809 15:39:00.094470 20549 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.3.89.193@KRBTEST.COM'} at 127.3.89.193:54213
I20260809 15:39:00.096797 20757 heartbeater.cc:503] Master 127.3.89.254:41873 was elected leader, sending a full tablet report...
Aug 09 15:39:00 dist-test-slave-2w3w krb5kdc[19918](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786289939, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.89.254@KRBTEST.COM for kudu/127.3.89.193@KRBTEST.COM
I20260809 15:39:00.104560 20767 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:39:00.093675 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42106 (local address 127.3.89.193:36955)
0809 15:39:00.094320 (+ 645us) server_negotiation.cc:206] Beginning negotiation
0809 15:39:00.094331 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0809 15:39:00.094352 (+ 21us) server_negotiation.cc:407] Connection header received
0809 15:39:00.094415 (+ 63us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 15:39:00.094420 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 15:39:00.094601 (+ 181us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 15:39:00.094750 (+ 149us) server_negotiation.cc:226] Negotiated authn=SASL
0809 15:39:00.095521 (+ 771us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:39:00.096690 (+ 1169us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:39:00.097777 (+ 1087us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:39:00.098242 (+ 465us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:39:00.098350 (+ 108us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:39:00.101128 (+ 2778us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 15:39:00.101136 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 15:39:00.101146 (+ 10us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 15:39:00.101168 (+ 22us) server_negotiation.cc:922] calling sasl_server_start()
0809 15:39:00.102664 (+ 1496us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 15:39:00.103235 (+ 571us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 15:39:00.103240 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 15:39:00.103241 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 15:39:00.103321 (+ 80us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 15:39:00.103609 (+ 288us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 15:39:00.103611 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 15:39:00.103612 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 15:39:00.103830 (+ 218us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 15:39:00.103928 (+ 98us) server_negotiation.cc:1035] Waiting for connection context
0809 15:39:00.104291 (+ 363us) server_negotiation.cc:299] Negotiation successful
0809 15:39:00.104426 (+ 135us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":542,"thread_start_us":117,"threads_started":1}
I20260809 15:39:00.104579 20766 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:39:00.093590 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.89.193:36955 (local address 127.0.0.1:42106)
0809 15:39:00.094001 (+ 411us) negotiation.cc:107] Waiting for socket to connect
0809 15:39:00.094048 (+ 47us) client_negotiation.cc:174] Beginning negotiation
0809 15:39:00.094182 (+ 134us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 15:39:00.094771 (+ 589us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 15:39:00.094777 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 15:39:00.095226 (+ 449us) client_negotiation.cc:189] Negotiated authn=SASL
0809 15:39:00.095388 (+ 162us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:39:00.095397 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:39:00.096823 (+ 1426us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:39:00.096826 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:39:00.097653 (+ 827us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:39:00.097662 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:39:00.098505 (+ 843us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:39:00.098511 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:39:00.098629 (+ 118us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:39:00.099198 (+ 569us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 15:39:00.099213 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0809 15:39:00.100986 (+ 1773us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 15:39:00.102778 (+ 1792us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 15:39:00.102787 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 15:39:00.102792 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0809 15:39:00.103128 (+ 336us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 15:39:00.103414 (+ 286us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 15:39:00.103417 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 15:39:00.103420 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0809 15:39:00.103517 (+ 97us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 15:39:00.103941 (+ 424us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 15:39:00.103946 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 15:39:00.104069 (+ 123us) client_negotiation.cc:769] Sending connection context
0809 15:39:00.104204 (+ 135us) client_negotiation.cc:240] Negotiation successful
0809 15:39:00.104400 (+ 196us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":230,"thread_start_us":101,"threads_started":1}
I20260809 15:39:00.105638 20685 tablet_service.cc:1514] Processing DeleteTablet for tablet f8a4a8a731934dbf8c0716edb5004f83 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 15:38:39 UTC) from {username='kudu', principal='kudu/127.3.89.254@KRBTEST.COM'} at 127.0.0.1:42106
I20260809 15:39:00.106488 20768 tablet_replica.cc:330] stopping tablet replica
I20260809 15:39:00.106783 20768 raft_consensus.cc:2229] T f8a4a8a731934dbf8c0716edb5004f83 P 23bd656811574169affd015915dd21e5 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 15:39:00.107050 20768 raft_consensus.cc:2258] T f8a4a8a731934dbf8c0716edb5004f83 P 23bd656811574169affd015915dd21e5 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 15:39:00.108104 20768 ts_tablet_manager.cc:1905] T f8a4a8a731934dbf8c0716edb5004f83 P 23bd656811574169affd015915dd21e5: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 15:39:00.112551 20768 ts_tablet_manager.cc:1918] T f8a4a8a731934dbf8c0716edb5004f83 P 23bd656811574169affd015915dd21e5: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 15:39:00.112653 20768 log.cc:1196] T f8a4a8a731934dbf8c0716edb5004f83 P 23bd656811574169affd015915dd21e5: Deleting WAL directory at /tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786289732823074-3431-0/minicluster-data/ts-0/wal/wals/f8a4a8a731934dbf8c0716edb5004f83
I20260809 15:39:00.112973 20768 ts_tablet_manager.cc:1939] T f8a4a8a731934dbf8c0716edb5004f83 P 23bd656811574169affd015915dd21e5: Deleting consensus metadata
I20260809 15:39:00.114952 20532 catalog_manager.cc:4841] TS 23bd656811574169affd015915dd21e5 (127.3.89.193:36955): tablet f8a4a8a731934dbf8c0716edb5004f83 (table test-table [id=4145f9c64db048f399b8c8e1804a7156]) successfully deleted
W20260809 15:39:00.192282 20764 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 15:39:00.192512 20764 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 15:39:00.192552 20764 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 15:39:00.196396 20764 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 15:39:00.196449 20764 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 15:39:00.196503 20764 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.3.89.194
I20260809 15:39:00.202072 20764 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.3.89.212:37503
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786289732823074-3431-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786289732823074-3431-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786289732823074-3431-0/krb5kdc/kudu_127.3.89.194.keytab
--principal=kudu/127.3.89.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.3.89.194:32773
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786289732823074-3431-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.3.89.194
--webserver_port=39463
--tserver_master_addrs=127.3.89.254:41873
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.20764
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.3.89.194
--log_dir=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786289732823074-3431-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 09 Aug 2026 14:46:55 UTC on ccycloud.quasar-yswkmf.root.comops.site
build id 4576540
I20260809 15:39:00.203034 20764 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 15:39:00.203967 20764 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 15:39:00.213191 20776 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 15:39:00.213187 20774 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 15:39:00.213165 20773 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 15:39:00.213380 20764 server_base.cc:1034] running on GCE node
I20260809 15:39:00.213863 20764 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 15:39:00.214545 20764 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 15:39:00.215728 20764 hybrid_clock.cc:648] HybridClock initialized: now 1786289940215708 us; error 42 us; skew 500 ppm
Aug 09 15:39:00 dist-test-slave-2w3w krb5kdc[19918](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786289940, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.89.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 15:39:00.218466 20764 init.cc:377] Logged in from keytab as kudu/127.3.89.194@KRBTEST.COM (short username kudu)
I20260809 15:39:00.219403 20764 webserver.cc:460] Webserver started at http://127.3.89.194:39463/ using document root <none> and password file <none>
I20260809 15:39:00.219959 20764 fs_manager.cc:359] Metadata directory not provided
I20260809 15:39:00.220057 20764 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 15:39:00.223786 20764 fs_manager.cc:709] Time spent opening directory manager: real 0.002s user 0.003s sys 0.000s
I20260809 15:39:00.225837 20784 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 15:39:00.227145 20764 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260809 15:39:00.227258 20764 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786289732823074-3431-0/minicluster-data/ts-1/data,/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786289732823074-3431-0/minicluster-data/ts-1/wal
uuid: "7ba1b776d4734e909008ca7c997fde5f"
format_stamp: "Formatted at 2026-08-09 15:38:39 on dist-test-slave-2w3w"
server_key: "05ddc3e034248186ff04a1599e9b0be6"
server_key_iv: "ac5b212a1983e3e9d8ae56c81442b5d1"
server_key_version: "encryptionkey@0"
I20260809 15:39:00.227777 20764 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786289732823074-3431-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786289732823074-3431-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786289732823074-3431-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 15:39:00.253293 20764 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 15:39:00.256736 20764 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 15:39:00.256889 20764 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 15:39:00.257501 20764 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 15:39:00.258287 20764 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 15:39:00.258334 20764 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 15:39:00.258400 20764 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 15:39:00.258436 20764 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 15:39:00.267396 20764 rpc_server.cc:307] RPC server started. Bound to: 127.3.89.194:32773
I20260809 15:39:00.267422 20909 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.3.89.194:32773 every 8 connection(s)
I20260809 15:39:00.268335 20764 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786289732823074-3431-0/minicluster-data/ts-1/data/info.pb
I20260809 15:39:00.271152 3431 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskm314YR/build/debug/bin/kudu as pid 20764
I20260809 15:39:00.271659 3431 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskm314YR/build/debug/bin/kudu
/tmp/dist-test-taskm314YR/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786289732823074-3431-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786289732823074-3431-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786289732823074-3431-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786289732823074-3431-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.3.89.195:44003
--local_ip_for_outbound_sockets=127.3.89.195
--tserver_master_addrs=127.3.89.254:41873
--webserver_port=43913
--webserver_interface=127.3.89.195
--keytab_file=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786289732823074-3431-0/krb5kdc/kudu_127.3.89.195.keytab
--principal=kudu/127.3.89.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.3.89.212:37503
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786289732823074-3431-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786289732823074-3431-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786289732823074-3431-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 09 15:39:00 dist-test-slave-2w3w krb5kdc[19918](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786289940, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.89.194@KRBTEST.COM for kudu/127.3.89.254@KRBTEST.COM
I20260809 15:39:00.283190 20761 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:39:00.270014 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.89.194:57129 (local address 127.3.89.254:41873)
0809 15:39:00.270158 (+ 144us) server_negotiation.cc:206] Beginning negotiation
0809 15:39:00.270161 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0809 15:39:00.271200 (+ 1039us) server_negotiation.cc:407] Connection header received
0809 15:39:00.272499 (+ 1299us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 15:39:00.272502 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 15:39:00.272549 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 15:39:00.272617 (+ 68us) server_negotiation.cc:226] Negotiated authn=SASL
0809 15:39:00.274085 (+ 1468us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:39:00.275008 (+ 923us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:39:00.276140 (+ 1132us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:39:00.276376 (+ 236us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:39:00.276520 (+ 144us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:39:00.279390 (+ 2870us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 15:39:00.279393 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 15:39:00.279395 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 15:39:00.279420 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0809 15:39:00.280901 (+ 1481us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 15:39:00.281548 (+ 647us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 15:39:00.281550 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 15:39:00.281551 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 15:39:00.281626 (+ 75us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 15:39:00.282025 (+ 399us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 15:39:00.282027 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 15:39:00.282028 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 15:39:00.282190 (+ 162us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 15:39:00.282266 (+ 76us) server_negotiation.cc:1035] Waiting for connection context
0809 15:39:00.282909 (+ 643us) server_negotiation.cc:299] Negotiation successful
0809 15:39:00.283067 (+ 158us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":57}
I20260809 15:39:00.284387 20912 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:39:00.270322 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.89.254:41873 (local address 127.3.89.194:57129)
0809 15:39:00.271021 (+ 699us) negotiation.cc:107] Waiting for socket to connect
0809 15:39:00.271082 (+ 61us) client_negotiation.cc:174] Beginning negotiation
0809 15:39:00.272262 (+ 1180us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 15:39:00.272811 (+ 549us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 15:39:00.272822 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 15:39:00.273322 (+ 500us) client_negotiation.cc:189] Negotiated authn=SASL
0809 15:39:00.273922 (+ 600us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:39:00.273937 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:39:00.275129 (+ 1192us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:39:00.275131 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:39:00.276012 (+ 881us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:39:00.276021 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:39:00.276561 (+ 540us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:39:00.276565 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:39:00.276726 (+ 161us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:39:00.277319 (+ 593us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 15:39:00.277335 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0809 15:39:00.279208 (+ 1873us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 15:39:00.281026 (+ 1818us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 15:39:00.281032 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 15:39:00.281048 (+ 16us) client_negotiation.cc:760] calling sasl_client_step()
0809 15:39:00.281447 (+ 399us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 15:39:00.281720 (+ 273us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 15:39:00.281726 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 15:39:00.281730 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0809 15:39:00.281925 (+ 195us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 15:39:00.282321 (+ 396us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 15:39:00.282331 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 15:39:00.282655 (+ 324us) client_negotiation.cc:769] Sending connection context
0809 15:39:00.282886 (+ 231us) client_negotiation.cc:240] Negotiation successful
0809 15:39:00.283272 (+ 386us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":447,"thread_start_us":109,"threads_started":1}
I20260809 15:39:00.285549 20910 heartbeater.cc:347] Connected to a master server at 127.3.89.254:41873
I20260809 15:39:00.285789 20910 heartbeater.cc:464] Registering TS with master...
I20260809 15:39:00.286276 20910 heartbeater.cc:511] Master 127.3.89.254:41873 requested a full tablet report, sending...
I20260809 15:39:00.287305 20549 ts_manager.cc:194] Registered new tserver with Master: 7ba1b776d4734e909008ca7c997fde5f (127.3.89.194:32773)
I20260809 15:39:00.287865 20549 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.3.89.194@KRBTEST.COM'} at 127.3.89.194:57129
W20260809 15:39:00.383620 20913 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 15:39:00.383891 20913 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 15:39:00.383968 20913 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 15:39:00.387890 20913 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 15:39:00.387945 20913 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 15:39:00.388063 20913 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.3.89.195
I20260809 15:39:00.394364 20913 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.3.89.212:37503
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786289732823074-3431-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786289732823074-3431-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786289732823074-3431-0/krb5kdc/kudu_127.3.89.195.keytab
--principal=kudu/127.3.89.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.3.89.195:44003
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786289732823074-3431-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.3.89.195
--webserver_port=43913
--tserver_master_addrs=127.3.89.254:41873
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.20913
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.3.89.195
--log_dir=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786289732823074-3431-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 09 Aug 2026 14:46:55 UTC on ccycloud.quasar-yswkmf.root.comops.site
build id 4576540
I20260809 15:39:00.395576 20913 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 15:39:00.396580 20913 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 15:39:00.406565 20919 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 15:39:00.406554 20918 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 15:39:00.406558 20921 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 15:39:00.406826 20913 server_base.cc:1034] running on GCE node
I20260809 15:39:00.407287 20913 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 15:39:00.407846 20913 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 15:39:00.409039 20913 hybrid_clock.cc:648] HybridClock initialized: now 1786289940409027 us; error 24 us; skew 500 ppm
Aug 09 15:39:00 dist-test-slave-2w3w krb5kdc[19918](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786289940, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.89.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 15:39:00.411808 20913 init.cc:377] Logged in from keytab as kudu/127.3.89.195@KRBTEST.COM (short username kudu)
I20260809 15:39:00.412628 20913 webserver.cc:460] Webserver started at http://127.3.89.195:43913/ using document root <none> and password file <none>
I20260809 15:39:00.413127 20913 fs_manager.cc:359] Metadata directory not provided
I20260809 15:39:00.413197 20913 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 15:39:00.417097 20913 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.003s sys 0.001s
I20260809 15:39:00.419025 20928 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 15:39:00.420333 20913 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.001s
I20260809 15:39:00.420480 20913 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786289732823074-3431-0/minicluster-data/ts-2/data,/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786289732823074-3431-0/minicluster-data/ts-2/wal
uuid: "440d47534a314ca88247d80185c766f8"
format_stamp: "Formatted at 2026-08-09 15:38:39 on dist-test-slave-2w3w"
server_key: "62d7b19b3c4c4511cb8d34b326b0568e"
server_key_iv: "1f6e4f692caf051f3d781a8ccd66e70b"
server_key_version: "encryptionkey@0"
I20260809 15:39:00.421069 20913 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786289732823074-3431-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786289732823074-3431-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786289732823074-3431-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 15:39:00.436915 20913 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 15:39:00.440555 20913 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 15:39:00.440789 20913 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 15:39:00.441432 20913 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 15:39:00.442318 20913 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 15:39:00.442369 20913 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 15:39:00.442446 20913 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 15:39:00.442492 20913 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 15:39:00.452332 20913 rpc_server.cc:307] RPC server started. Bound to: 127.3.89.195:44003
I20260809 15:39:00.452353 21050 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.3.89.195:44003 every 8 connection(s)
I20260809 15:39:00.453392 20913 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786289732823074-3431-0/minicluster-data/ts-2/data/info.pb
I20260809 15:39:00.459429 3431 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskm314YR/build/debug/bin/kudu as pid 20913
Aug 09 15:39:00 dist-test-slave-2w3w krb5kdc[19918](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786289940, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.89.195@KRBTEST.COM for kudu/127.3.89.254@KRBTEST.COM
I20260809 15:39:00.467751 20761 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:39:00.455381 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.89.195:56963 (local address 127.3.89.254:41873)
0809 15:39:00.455560 (+ 179us) server_negotiation.cc:206] Beginning negotiation
0809 15:39:00.455566 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 15:39:00.456635 (+ 1069us) server_negotiation.cc:407] Connection header received
0809 15:39:00.457440 (+ 805us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 15:39:00.457443 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 15:39:00.457492 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 15:39:00.457606 (+ 114us) server_negotiation.cc:226] Negotiated authn=SASL
0809 15:39:00.458676 (+ 1070us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:39:00.459331 (+ 655us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:39:00.460907 (+ 1576us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:39:00.461331 (+ 424us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:39:00.461467 (+ 136us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:39:00.464224 (+ 2757us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 15:39:00.464227 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 15:39:00.464229 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 15:39:00.464251 (+ 22us) server_negotiation.cc:922] calling sasl_server_start()
0809 15:39:00.465616 (+ 1365us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 15:39:00.466170 (+ 554us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 15:39:00.466180 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 15:39:00.466182 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 15:39:00.466254 (+ 72us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 15:39:00.466587 (+ 333us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 15:39:00.466591 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 15:39:00.466593 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 15:39:00.466763 (+ 170us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 15:39:00.466871 (+ 108us) server_negotiation.cc:1035] Waiting for connection context
0809 15:39:00.467413 (+ 542us) server_negotiation.cc:299] Negotiation successful
0809 15:39:00.467582 (+ 169us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":70}
I20260809 15:39:00.468660 21053 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:39:00.455813 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.89.254:41873 (local address 127.3.89.195:56963)
0809 15:39:00.456473 (+ 660us) negotiation.cc:107] Waiting for socket to connect
0809 15:39:00.456513 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0809 15:39:00.457246 (+ 733us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 15:39:00.457763 (+ 517us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 15:39:00.457770 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 15:39:00.458101 (+ 331us) client_negotiation.cc:189] Negotiated authn=SASL
0809 15:39:00.458548 (+ 447us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:39:00.458561 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:39:00.459475 (+ 914us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:39:00.459481 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:39:00.460782 (+ 1301us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:39:00.460794 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:39:00.461487 (+ 693us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:39:00.461489 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:39:00.461637 (+ 148us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:39:00.462237 (+ 600us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 15:39:00.462256 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0809 15:39:00.464056 (+ 1800us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 15:39:00.465736 (+ 1680us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 15:39:00.465742 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 15:39:00.465755 (+ 13us) client_negotiation.cc:760] calling sasl_client_step()
0809 15:39:00.466047 (+ 292us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 15:39:00.466346 (+ 299us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 15:39:00.466349 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 15:39:00.466351 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 15:39:00.466490 (+ 139us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 15:39:00.466869 (+ 379us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 15:39:00.466876 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 15:39:00.467185 (+ 309us) client_negotiation.cc:769] Sending connection context
0809 15:39:00.467369 (+ 184us) client_negotiation.cc:240] Negotiation successful
0809 15:39:00.467676 (+ 307us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":479,"thread_start_us":136,"threads_started":1}
I20260809 15:39:00.469890 21055 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:39:00.460141 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40948 (local address 127.3.89.254:41873)
0809 15:39:00.460367 (+ 226us) server_negotiation.cc:206] Beginning negotiation
0809 15:39:00.460372 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 15:39:00.460549 (+ 177us) server_negotiation.cc:407] Connection header received
0809 15:39:00.460649 (+ 100us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 15:39:00.460652 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 15:39:00.460702 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 15:39:00.460800 (+ 98us) server_negotiation.cc:226] Negotiated authn=SASL
0809 15:39:00.461807 (+ 1007us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:39:00.462572 (+ 765us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:39:00.463928 (+ 1356us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:39:00.464172 (+ 244us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:39:00.464286 (+ 114us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:39:00.465867 (+ 1581us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 15:39:00.465890 (+ 23us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 15:39:00.465893 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 15:39:00.465924 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0809 15:39:00.467583 (+ 1659us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 15:39:00.468120 (+ 537us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 15:39:00.468124 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 15:39:00.468126 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 15:39:00.468212 (+ 86us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 15:39:00.468501 (+ 289us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 15:39:00.468504 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 15:39:00.468506 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 15:39:00.468715 (+ 209us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 15:39:00.469014 (+ 299us) server_negotiation.cc:1035] Waiting for connection context
0809 15:39:00.469572 (+ 558us) server_negotiation.cc:299] Negotiation successful
0809 15:39:00.469705 (+ 133us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":167,"thread_start_us":84,"threads_started":1}
I20260809 15:39:00.469928 21051 heartbeater.cc:347] Connected to a master server at 127.3.89.254:41873
I20260809 15:39:00.470171 21051 heartbeater.cc:464] Registering TS with master...
I20260809 15:39:00.470745 21051 heartbeater.cc:511] Master 127.3.89.254:41873 requested a full tablet report, sending...
I20260809 15:39:00.471829 20548 ts_manager.cc:194] Registered new tserver with Master: 440d47534a314ca88247d80185c766f8 (127.3.89.195:44003)
I20260809 15:39:00.472396 20548 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.3.89.195@KRBTEST.COM'} at 127.3.89.195:56963
I20260809 15:39:00.483621 3431 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260809 15:39:00.487586 21055 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:39:00.484196 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40952 (local address 127.3.89.254:41873)
0809 15:39:00.484316 (+ 120us) server_negotiation.cc:206] Beginning negotiation
0809 15:39:00.484319 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0809 15:39:00.484532 (+ 213us) server_negotiation.cc:407] Connection header received
0809 15:39:00.484664 (+ 132us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 15:39:00.484666 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 15:39:00.484749 (+ 83us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 15:39:00.484831 (+ 82us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 15:39:00.485042 (+ 211us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:39:00.485547 (+ 505us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:39:00.486555 (+ 1008us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:39:00.486780 (+ 225us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:39:00.486852 (+ 72us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:39:00.487175 (+ 323us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 15:39:00.487294 (+ 119us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_AUTHENTICATION_TOKEN: Not authorized: token expired
0809 15:39:00.487498 (+ 204us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:40952: token expired
Metrics: {"server-negotiator.queue_time_us":48}
W20260809 15:39:00.487669 21055 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:40952: token expired
W20260809 15:39:00.488039 20471 master_proxy_rpc.cc:203] Re-attempting CreateTable request to leader Master (127.3.89.254:41873)
I20260809 15:39:00.494925 21055 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:39:00.488442 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40960 (local address 127.3.89.254:41873)
0809 15:39:00.488560 (+ 118us) server_negotiation.cc:206] Beginning negotiation
0809 15:39:00.488564 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 15:39:00.488583 (+ 19us) server_negotiation.cc:407] Connection header received
0809 15:39:00.488677 (+ 94us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 15:39:00.488679 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 15:39:00.488712 (+ 33us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 15:39:00.488776 (+ 64us) server_negotiation.cc:226] Negotiated authn=SASL
0809 15:39:00.489382 (+ 606us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:39:00.489828 (+ 446us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:39:00.490619 (+ 791us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:39:00.490810 (+ 191us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:39:00.490883 (+ 73us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:39:00.492219 (+ 1336us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 15:39:00.492222 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 15:39:00.492224 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 15:39:00.492244 (+ 20us) server_negotiation.cc:922] calling sasl_server_start()
0809 15:39:00.493561 (+ 1317us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 15:39:00.493954 (+ 393us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 15:39:00.493956 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 15:39:00.493957 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 15:39:00.494040 (+ 83us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 15:39:00.494274 (+ 234us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 15:39:00.494276 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 15:39:00.494277 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 15:39:00.494424 (+ 147us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 15:39:00.494519 (+ 95us) server_negotiation.cc:1035] Waiting for connection context
0809 15:39:00.494733 (+ 214us) server_negotiation.cc:299] Negotiation successful
0809 15:39:00.494823 (+ 90us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":54}
I20260809 15:39:00.497273 20548 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:40960:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260809 15:39:00.500917 20548 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260809 15:39:00.508792 20685 tablet_service.cc:1467] Processing CreateTablet for tablet 1746f14a92d5425d935e609571748e70 (DEFAULT_TABLE table=test-table [id=88f77c30fbe94b929b26dd067b365dde]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 15:39:00.509498 20685 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 1746f14a92d5425d935e609571748e70. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 15:39:00.512241 20637 tablet_bootstrap.cc:492] T 1746f14a92d5425d935e609571748e70 P 23bd656811574169affd015915dd21e5: Bootstrap starting.
I20260809 15:39:00.513048 20637 tablet_bootstrap.cc:654] T 1746f14a92d5425d935e609571748e70 P 23bd656811574169affd015915dd21e5: Neither blocks nor log segments found. Creating new log.
I20260809 15:39:00.513742 20766 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:39:00.507626 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.89.194:32773 (local address 127.0.0.1:57902)
0809 15:39:00.507747 (+ 121us) negotiation.cc:107] Waiting for socket to connect
0809 15:39:00.507763 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0809 15:39:00.507844 (+ 81us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 15:39:00.508794 (+ 950us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 15:39:00.508797 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 15:39:00.508820 (+ 23us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 15:39:00.508917 (+ 97us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:39:00.508923 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:39:00.510624 (+ 1701us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:39:00.510628 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:39:00.511882 (+ 1254us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:39:00.511896 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:39:00.513209 (+ 1313us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:39:00.513213 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:39:00.513342 (+ 129us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:39:00.513400 (+ 58us) client_negotiation.cc:769] Sending connection context
0809 15:39:00.513458 (+ 58us) client_negotiation.cc:240] Negotiation successful
0809 15:39:00.513558 (+ 100us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":47,"mutex_wait_us":9}
I20260809 15:39:00.514047 21062 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:39:00.508445 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46246 (local address 127.3.89.195:44003)
0809 15:39:00.508865 (+ 420us) server_negotiation.cc:206] Beginning negotiation
0809 15:39:00.508879 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0809 15:39:00.508906 (+ 27us) server_negotiation.cc:407] Connection header received
0809 15:39:00.509042 (+ 136us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 15:39:00.509048 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 15:39:00.509278 (+ 230us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 15:39:00.509454 (+ 176us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 15:39:00.509685 (+ 231us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:39:00.510801 (+ 1116us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:39:00.512286 (+ 1485us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:39:00.513202 (+ 916us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:39:00.513313 (+ 111us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:39:00.513472 (+ 159us) server_negotiation.cc:1035] Waiting for connection context
0809 15:39:00.513666 (+ 194us) server_negotiation.cc:299] Negotiation successful
0809 15:39:00.513900 (+ 234us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":318,"thread_start_us":90,"threads_started":1}
I20260809 15:39:00.514379 21061 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:39:00.508000 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.89.195:44003 (local address 127.0.0.1:46246)
0809 15:39:00.508483 (+ 483us) negotiation.cc:107] Waiting for socket to connect
0809 15:39:00.508499 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0809 15:39:00.508589 (+ 90us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 15:39:00.509445 (+ 856us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 15:39:00.509448 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 15:39:00.509475 (+ 27us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 15:39:00.509592 (+ 117us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:39:00.509599 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:39:00.510927 (+ 1328us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:39:00.510934 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:39:00.512132 (+ 1198us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:39:00.512144 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:39:00.513334 (+ 1190us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:39:00.513340 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:39:00.513482 (+ 142us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:39:00.513520 (+ 38us) client_negotiation.cc:769] Sending connection context
0809 15:39:00.513613 (+ 93us) client_negotiation.cc:240] Negotiation successful
0809 15:39:00.513682 (+ 69us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":417,"thread_start_us":114,"threads_started":1}
I20260809 15:39:00.514931 20637 tablet_bootstrap.cc:492] T 1746f14a92d5425d935e609571748e70 P 23bd656811574169affd015915dd21e5: No bootstrap required, opened a new log
I20260809 15:39:00.515051 21060 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:39:00.507844 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57902 (local address 127.3.89.194:32773)
0809 15:39:00.508207 (+ 363us) server_negotiation.cc:206] Beginning negotiation
0809 15:39:00.508225 (+ 18us) server_negotiation.cc:399] Waiting for connection header
0809 15:39:00.508251 (+ 26us) server_negotiation.cc:407] Connection header received
0809 15:39:00.508339 (+ 88us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 15:39:00.508344 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 15:39:00.508590 (+ 246us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 15:39:00.508825 (+ 235us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 15:39:00.509114 (+ 289us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:39:00.510490 (+ 1376us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:39:00.512030 (+ 1540us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:39:00.513046 (+ 1016us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:39:00.514524 (+ 1478us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:39:00.514693 (+ 169us) server_negotiation.cc:1035] Waiting for connection context
0809 15:39:00.514773 (+ 80us) server_negotiation.cc:299] Negotiation successful
0809 15:39:00.514872 (+ 99us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":258,"thread_start_us":170,"threads_started":1}
I20260809 15:39:00.515074 20637 ts_tablet_manager.cc:1397] T 1746f14a92d5425d935e609571748e70 P 23bd656811574169affd015915dd21e5: Time spent bootstrapping tablet: real 0.003s user 0.001s sys 0.001s
I20260809 15:39:00.515542 20637 raft_consensus.cc:348] T 1746f14a92d5425d935e609571748e70 P 23bd656811574169affd015915dd21e5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7ba1b776d4734e909008ca7c997fde5f" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 32773 } } peers { permanent_uuid: "440d47534a314ca88247d80185c766f8" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 44003 } } peers { permanent_uuid: "23bd656811574169affd015915dd21e5" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 36955 } }
I20260809 15:39:00.515676 20637 raft_consensus.cc:374] T 1746f14a92d5425d935e609571748e70 P 23bd656811574169affd015915dd21e5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 15:39:00.515714 20637 raft_consensus.cc:729] T 1746f14a92d5425d935e609571748e70 P 23bd656811574169affd015915dd21e5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 23bd656811574169affd015915dd21e5, State: Initialized, Role: FOLLOWER
I20260809 15:39:00.515839 20637 consensus_queue.cc:260] T 1746f14a92d5425d935e609571748e70 P 23bd656811574169affd015915dd21e5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7ba1b776d4734e909008ca7c997fde5f" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 32773 } } peers { permanent_uuid: "440d47534a314ca88247d80185c766f8" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 44003 } } peers { permanent_uuid: "23bd656811574169affd015915dd21e5" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 36955 } }
I20260809 15:39:00.516098 20637 ts_tablet_manager.cc:1428] T 1746f14a92d5425d935e609571748e70 P 23bd656811574169affd015915dd21e5: Time spent starting tablet: real 0.001s user 0.000s sys 0.001s
I20260809 15:39:00.517764 20979 tablet_service.cc:1467] Processing CreateTablet for tablet 1746f14a92d5425d935e609571748e70 (DEFAULT_TABLE table=test-table [id=88f77c30fbe94b929b26dd067b365dde]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 15:39:00.517849 20834 tablet_service.cc:1467] Processing CreateTablet for tablet 1746f14a92d5425d935e609571748e70 (DEFAULT_TABLE table=test-table [id=88f77c30fbe94b929b26dd067b365dde]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 15:39:00.518790 20979 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 1746f14a92d5425d935e609571748e70. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 15:39:00.518873 20834 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 1746f14a92d5425d935e609571748e70. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 15:39:00.525355 21064 tablet_bootstrap.cc:492] T 1746f14a92d5425d935e609571748e70 P 440d47534a314ca88247d80185c766f8: Bootstrap starting.
I20260809 15:39:00.526026 21065 tablet_bootstrap.cc:492] T 1746f14a92d5425d935e609571748e70 P 7ba1b776d4734e909008ca7c997fde5f: Bootstrap starting.
I20260809 15:39:00.527896 21064 tablet_bootstrap.cc:654] T 1746f14a92d5425d935e609571748e70 P 440d47534a314ca88247d80185c766f8: Neither blocks nor log segments found. Creating new log.
I20260809 15:39:00.528429 21065 tablet_bootstrap.cc:654] T 1746f14a92d5425d935e609571748e70 P 7ba1b776d4734e909008ca7c997fde5f: Neither blocks nor log segments found. Creating new log.
I20260809 15:39:00.528939 21064 log.cc:824] T 1746f14a92d5425d935e609571748e70 P 440d47534a314ca88247d80185c766f8: Log is configured to *not* fsync() on all Append() calls
I20260809 15:39:00.529136 21065 log.cc:824] T 1746f14a92d5425d935e609571748e70 P 7ba1b776d4734e909008ca7c997fde5f: Log is configured to *not* fsync() on all Append() calls
I20260809 15:39:00.530943 21064 tablet_bootstrap.cc:492] T 1746f14a92d5425d935e609571748e70 P 440d47534a314ca88247d80185c766f8: No bootstrap required, opened a new log
I20260809 15:39:00.531066 21065 tablet_bootstrap.cc:492] T 1746f14a92d5425d935e609571748e70 P 7ba1b776d4734e909008ca7c997fde5f: No bootstrap required, opened a new log
I20260809 15:39:00.531217 21064 ts_tablet_manager.cc:1397] T 1746f14a92d5425d935e609571748e70 P 440d47534a314ca88247d80185c766f8: Time spent bootstrapping tablet: real 0.006s user 0.004s sys 0.001s
I20260809 15:39:00.531247 21065 ts_tablet_manager.cc:1397] T 1746f14a92d5425d935e609571748e70 P 7ba1b776d4734e909008ca7c997fde5f: Time spent bootstrapping tablet: real 0.006s user 0.005s sys 0.000s
I20260809 15:39:00.534198 21064 raft_consensus.cc:348] T 1746f14a92d5425d935e609571748e70 P 440d47534a314ca88247d80185c766f8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7ba1b776d4734e909008ca7c997fde5f" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 32773 } } peers { permanent_uuid: "440d47534a314ca88247d80185c766f8" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 44003 } } peers { permanent_uuid: "23bd656811574169affd015915dd21e5" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 36955 } }
I20260809 15:39:00.534423 21064 raft_consensus.cc:374] T 1746f14a92d5425d935e609571748e70 P 440d47534a314ca88247d80185c766f8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 15:39:00.534458 21064 raft_consensus.cc:729] T 1746f14a92d5425d935e609571748e70 P 440d47534a314ca88247d80185c766f8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 440d47534a314ca88247d80185c766f8, State: Initialized, Role: FOLLOWER
I20260809 15:39:00.534351 21065 raft_consensus.cc:348] T 1746f14a92d5425d935e609571748e70 P 7ba1b776d4734e909008ca7c997fde5f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7ba1b776d4734e909008ca7c997fde5f" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 32773 } } peers { permanent_uuid: "440d47534a314ca88247d80185c766f8" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 44003 } } peers { permanent_uuid: "23bd656811574169affd015915dd21e5" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 36955 } }
I20260809 15:39:00.534693 21065 raft_consensus.cc:374] T 1746f14a92d5425d935e609571748e70 P 7ba1b776d4734e909008ca7c997fde5f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 15:39:00.534804 21065 raft_consensus.cc:729] T 1746f14a92d5425d935e609571748e70 P 7ba1b776d4734e909008ca7c997fde5f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7ba1b776d4734e909008ca7c997fde5f, State: Initialized, Role: FOLLOWER
I20260809 15:39:00.535005 21064 consensus_queue.cc:260] T 1746f14a92d5425d935e609571748e70 P 440d47534a314ca88247d80185c766f8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7ba1b776d4734e909008ca7c997fde5f" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 32773 } } peers { permanent_uuid: "440d47534a314ca88247d80185c766f8" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 44003 } } peers { permanent_uuid: "23bd656811574169affd015915dd21e5" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 36955 } }
I20260809 15:39:00.535284 21065 consensus_queue.cc:260] T 1746f14a92d5425d935e609571748e70 P 7ba1b776d4734e909008ca7c997fde5f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7ba1b776d4734e909008ca7c997fde5f" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 32773 } } peers { permanent_uuid: "440d47534a314ca88247d80185c766f8" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 44003 } } peers { permanent_uuid: "23bd656811574169affd015915dd21e5" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 36955 } }
I20260809 15:39:00.535724 21051 heartbeater.cc:503] Master 127.3.89.254:41873 was elected leader, sending a full tablet report...
I20260809 15:39:00.536016 21064 ts_tablet_manager.cc:1428] T 1746f14a92d5425d935e609571748e70 P 440d47534a314ca88247d80185c766f8: Time spent starting tablet: real 0.005s user 0.002s sys 0.002s
I20260809 15:39:00.536253 21065 ts_tablet_manager.cc:1428] T 1746f14a92d5425d935e609571748e70 P 7ba1b776d4734e909008ca7c997fde5f: Time spent starting tablet: real 0.005s user 0.005s sys 0.000s
I20260809 15:39:00.536603 20910 heartbeater.cc:503] Master 127.3.89.254:41873 was elected leader, sending a full tablet report...
W20260809 15:39:00.566337 20758 tablet.cc:2369] T 1746f14a92d5425d935e609571748e70 P 23bd656811574169affd015915dd21e5: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260809 15:39:00.704474 21052 tablet.cc:2369] T 1746f14a92d5425d935e609571748e70 P 440d47534a314ca88247d80185c766f8: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260809 15:39:00.769438 20911 tablet.cc:2369] T 1746f14a92d5425d935e609571748e70 P 7ba1b776d4734e909008ca7c997fde5f: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260809 15:39:00.831930 21068 raft_consensus.cc:482] T 1746f14a92d5425d935e609571748e70 P 440d47534a314ca88247d80185c766f8 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 15:39:00.832182 21068 raft_consensus.cc:504] T 1746f14a92d5425d935e609571748e70 P 440d47534a314ca88247d80185c766f8 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7ba1b776d4734e909008ca7c997fde5f" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 32773 } } peers { permanent_uuid: "440d47534a314ca88247d80185c766f8" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 44003 } } peers { permanent_uuid: "23bd656811574169affd015915dd21e5" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 36955 } }
I20260809 15:39:00.833413 21068 leader_election.cc:288] T 1746f14a92d5425d935e609571748e70 P 440d47534a314ca88247d80185c766f8 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7ba1b776d4734e909008ca7c997fde5f (127.3.89.194:32773), 23bd656811574169affd015915dd21e5 (127.3.89.193:36955)
I20260809 15:39:00.838577 21071 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:39:00.834183 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.89.195:40837 (local address 127.3.89.193:36955)
0809 15:39:00.834492 (+ 309us) server_negotiation.cc:206] Beginning negotiation
0809 15:39:00.834495 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0809 15:39:00.834509 (+ 14us) server_negotiation.cc:407] Connection header received
0809 15:39:00.834560 (+ 51us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 15:39:00.834563 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 15:39:00.834613 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 15:39:00.834775 (+ 162us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 15:39:00.835031 (+ 256us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:39:00.835847 (+ 816us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:39:00.837189 (+ 1342us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:39:00.837926 (+ 737us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:39:00.838045 (+ 119us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:39:00.838184 (+ 139us) server_negotiation.cc:1035] Waiting for connection context
0809 15:39:00.838355 (+ 171us) server_negotiation.cc:299] Negotiation successful
0809 15:39:00.838434 (+ 79us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":227,"thread_start_us":94,"threads_started":1}
I20260809 15:39:00.838572 21060 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:39:00.833787 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.89.195:49143 (local address 127.3.89.194:32773)
0809 15:39:00.833934 (+ 147us) server_negotiation.cc:206] Beginning negotiation
0809 15:39:00.833938 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 15:39:00.833955 (+ 17us) server_negotiation.cc:407] Connection header received
0809 15:39:00.834111 (+ 156us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 15:39:00.834114 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 15:39:00.834172 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 15:39:00.834292 (+ 120us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 15:39:00.834746 (+ 454us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:39:00.835669 (+ 923us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:39:00.837189 (+ 1520us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:39:00.837843 (+ 654us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:39:00.837994 (+ 151us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:39:00.838055 (+ 61us) server_negotiation.cc:1035] Waiting for connection context
0809 15:39:00.838356 (+ 301us) server_negotiation.cc:299] Negotiation successful
0809 15:39:00.838433 (+ 77us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":60}
I20260809 15:39:00.838591 21053 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:39:00.833696 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.89.194:32773 (local address 127.3.89.195:49143)
0809 15:39:00.833824 (+ 128us) negotiation.cc:107] Waiting for socket to connect
0809 15:39:00.833841 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0809 15:39:00.833981 (+ 140us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 15:39:00.834363 (+ 382us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 15:39:00.834366 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 15:39:00.834406 (+ 40us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 15:39:00.834588 (+ 182us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:39:00.834595 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:39:00.835788 (+ 1193us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:39:00.835790 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:39:00.837039 (+ 1249us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:39:00.837050 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:39:00.837966 (+ 916us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:39:00.837969 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:39:00.838123 (+ 154us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:39:00.838234 (+ 111us) client_negotiation.cc:769] Sending connection context
0809 15:39:00.838314 (+ 80us) client_negotiation.cc:240] Negotiation successful
0809 15:39:00.838402 (+ 88us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":45,"mutex_wait_us":64,"spinlock_wait_cycles":5376}
I20260809 15:39:00.838603 21070 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:39:00.834055 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.89.193:36955 (local address 127.3.89.195:40837)
0809 15:39:00.834369 (+ 314us) negotiation.cc:107] Waiting for socket to connect
0809 15:39:00.834383 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0809 15:39:00.834490 (+ 107us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 15:39:00.834775 (+ 285us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 15:39:00.834777 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 15:39:00.834792 (+ 15us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 15:39:00.834880 (+ 88us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:39:00.834884 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:39:00.835940 (+ 1056us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:39:00.835943 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:39:00.837064 (+ 1121us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:39:00.837071 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:39:00.838031 (+ 960us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:39:00.838034 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:39:00.838176 (+ 142us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:39:00.838237 (+ 61us) client_negotiation.cc:769] Sending connection context
0809 15:39:00.838314 (+ 77us) client_negotiation.cc:240] Negotiation successful
0809 15:39:00.838402 (+ 88us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":253,"mutex_wait_us":66,"spinlock_wait_cycles":49920,"thread_start_us":129,"threads_started":1}
I20260809 15:39:00.839267 20856 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1746f14a92d5425d935e609571748e70" candidate_uuid: "440d47534a314ca88247d80185c766f8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7ba1b776d4734e909008ca7c997fde5f" is_pre_election: true
I20260809 15:39:00.839586 20856 raft_consensus.cc:2454] T 1746f14a92d5425d935e609571748e70 P 7ba1b776d4734e909008ca7c997fde5f [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 440d47534a314ca88247d80185c766f8 in term 0.
I20260809 15:39:00.839929 20706 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1746f14a92d5425d935e609571748e70" candidate_uuid: "440d47534a314ca88247d80185c766f8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "23bd656811574169affd015915dd21e5" is_pre_election: true
I20260809 15:39:00.840143 20930 leader_election.cc:302] T 1746f14a92d5425d935e609571748e70 P 440d47534a314ca88247d80185c766f8 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 440d47534a314ca88247d80185c766f8, 7ba1b776d4734e909008ca7c997fde5f; no voters:
I20260809 15:39:00.840233 20706 raft_consensus.cc:2454] T 1746f14a92d5425d935e609571748e70 P 23bd656811574169affd015915dd21e5 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 440d47534a314ca88247d80185c766f8 in term 0.
I20260809 15:39:00.840473 21068 raft_consensus.cc:2781] T 1746f14a92d5425d935e609571748e70 P 440d47534a314ca88247d80185c766f8 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260809 15:39:00.840574 21068 raft_consensus.cc:482] T 1746f14a92d5425d935e609571748e70 P 440d47534a314ca88247d80185c766f8 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 15:39:00.840636 21068 raft_consensus.cc:3037] T 1746f14a92d5425d935e609571748e70 P 440d47534a314ca88247d80185c766f8 [term 0 FOLLOWER]: Advancing to term 1
I20260809 15:39:00.841650 21068 raft_consensus.cc:504] T 1746f14a92d5425d935e609571748e70 P 440d47534a314ca88247d80185c766f8 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7ba1b776d4734e909008ca7c997fde5f" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 32773 } } peers { permanent_uuid: "440d47534a314ca88247d80185c766f8" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 44003 } } peers { permanent_uuid: "23bd656811574169affd015915dd21e5" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 36955 } }
I20260809 15:39:00.841988 21068 leader_election.cc:288] T 1746f14a92d5425d935e609571748e70 P 440d47534a314ca88247d80185c766f8 [CANDIDATE]: Term 1 election: Requested vote from peers 7ba1b776d4734e909008ca7c997fde5f (127.3.89.194:32773), 23bd656811574169affd015915dd21e5 (127.3.89.193:36955)
I20260809 15:39:00.842375 20856 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1746f14a92d5425d935e609571748e70" candidate_uuid: "440d47534a314ca88247d80185c766f8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7ba1b776d4734e909008ca7c997fde5f"
I20260809 15:39:00.842525 20856 raft_consensus.cc:3037] T 1746f14a92d5425d935e609571748e70 P 7ba1b776d4734e909008ca7c997fde5f [term 0 FOLLOWER]: Advancing to term 1
I20260809 15:39:00.842549 20706 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1746f14a92d5425d935e609571748e70" candidate_uuid: "440d47534a314ca88247d80185c766f8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "23bd656811574169affd015915dd21e5"
I20260809 15:39:00.842691 20706 raft_consensus.cc:3037] T 1746f14a92d5425d935e609571748e70 P 23bd656811574169affd015915dd21e5 [term 0 FOLLOWER]: Advancing to term 1
I20260809 15:39:00.843663 20856 raft_consensus.cc:2454] T 1746f14a92d5425d935e609571748e70 P 7ba1b776d4734e909008ca7c997fde5f [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 440d47534a314ca88247d80185c766f8 in term 1.
I20260809 15:39:00.843905 20706 raft_consensus.cc:2454] T 1746f14a92d5425d935e609571748e70 P 23bd656811574169affd015915dd21e5 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 440d47534a314ca88247d80185c766f8 in term 1.
I20260809 15:39:00.844023 20930 leader_election.cc:302] T 1746f14a92d5425d935e609571748e70 P 440d47534a314ca88247d80185c766f8 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 440d47534a314ca88247d80185c766f8, 7ba1b776d4734e909008ca7c997fde5f; no voters:
I20260809 15:39:00.844285 21068 raft_consensus.cc:2781] T 1746f14a92d5425d935e609571748e70 P 440d47534a314ca88247d80185c766f8 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 15:39:00.844455 21068 raft_consensus.cc:686] T 1746f14a92d5425d935e609571748e70 P 440d47534a314ca88247d80185c766f8 [term 1 LEADER]: Becoming Leader. State: Replica: 440d47534a314ca88247d80185c766f8, State: Running, Role: LEADER
I20260809 15:39:00.844767 21068 consensus_queue.cc:237] T 1746f14a92d5425d935e609571748e70 P 440d47534a314ca88247d80185c766f8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7ba1b776d4734e909008ca7c997fde5f" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 32773 } } peers { permanent_uuid: "440d47534a314ca88247d80185c766f8" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 44003 } } peers { permanent_uuid: "23bd656811574169affd015915dd21e5" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 36955 } }
I20260809 15:39:00.847839 20547 catalog_manager.cc:5495] T 1746f14a92d5425d935e609571748e70 P 440d47534a314ca88247d80185c766f8 reported cstate change: term changed from 0 to 1, leader changed from <none> to 440d47534a314ca88247d80185c766f8 (127.3.89.195). New cstate: current_term: 1 leader_uuid: "440d47534a314ca88247d80185c766f8" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7ba1b776d4734e909008ca7c997fde5f" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 32773 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "440d47534a314ca88247d80185c766f8" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 44003 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "23bd656811574169affd015915dd21e5" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 36955 } health_report { overall_health: UNKNOWN } } }
I20260809 15:39:00.877269 21062 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:39:00.874147 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46252 (local address 127.3.89.195:44003)
0809 15:39:00.874280 (+ 133us) server_negotiation.cc:206] Beginning negotiation
0809 15:39:00.874283 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0809 15:39:00.874297 (+ 14us) server_negotiation.cc:407] Connection header received
0809 15:39:00.874423 (+ 126us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 15:39:00.874426 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 15:39:00.874486 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 15:39:00.874572 (+ 86us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 15:39:00.874804 (+ 232us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:39:00.875308 (+ 504us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:39:00.876084 (+ 776us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:39:00.876281 (+ 197us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:39:00.876354 (+ 73us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:39:00.876625 (+ 271us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 15:39:00.876979 (+ 354us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 15:39:00.877064 (+ 85us) server_negotiation.cc:1035] Waiting for connection context
0809 15:39:00.877125 (+ 61us) server_negotiation.cc:299] Negotiation successful
0809 15:39:00.877171 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":56}
I20260809 15:39:00.882675 20706 raft_consensus.cc:1261] T 1746f14a92d5425d935e609571748e70 P 23bd656811574169affd015915dd21e5 [term 1 FOLLOWER]: Refusing update from remote peer 440d47534a314ca88247d80185c766f8: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 15:39:00.882720 20856 raft_consensus.cc:1261] T 1746f14a92d5425d935e609571748e70 P 7ba1b776d4734e909008ca7c997fde5f [term 1 FOLLOWER]: Refusing update from remote peer 440d47534a314ca88247d80185c766f8: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 15:39:00.883437 21068 consensus_queue.cc:1035] T 1746f14a92d5425d935e609571748e70 P 440d47534a314ca88247d80185c766f8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7ba1b776d4734e909008ca7c997fde5f" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 32773 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 15:39:00.883718 21072 consensus_queue.cc:1035] T 1746f14a92d5425d935e609571748e70 P 440d47534a314ca88247d80185c766f8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "23bd656811574169affd015915dd21e5" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 36955 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 15:39:00.888994 21075 mvcc.cc:205] Tried to move back new op lower bound from 7316643597848653824 to 7316643597702430720. Current Snapshot: MvccSnapshot[applied={T|T < 7316643597848653824}]
I20260809 15:39:00.889418 21076 mvcc.cc:205] Tried to move back new op lower bound from 7316643597848653824 to 7316643597702430720. Current Snapshot: MvccSnapshot[applied={T|T < 7316643597848653824}]
I20260809 15:39:00.891100 21077 mvcc.cc:205] Tried to move back new op lower bound from 7316643597848653824 to 7316643597702430720. Current Snapshot: MvccSnapshot[applied={T|T < 7316643597848653824}]
I20260809 15:39:00.897884 20547 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:40960:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 15:39:00.898068 20547 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:40960:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 15:39:00.900545 20547 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 210f7703a6ee4761acec0985806eb015: Sending DeleteTablet for 3 replicas of tablet 1746f14a92d5425d935e609571748e70
I20260809 15:39:00.901160 20685 tablet_service.cc:1514] Processing DeleteTablet for tablet 1746f14a92d5425d935e609571748e70 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 15:39:00 UTC) from {username='kudu', principal='kudu/127.3.89.254@KRBTEST.COM'} at 127.0.0.1:42106
I20260809 15:39:00.901156 20834 tablet_service.cc:1514] Processing DeleteTablet for tablet 1746f14a92d5425d935e609571748e70 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 15:39:00 UTC) from {username='kudu', principal='kudu/127.3.89.254@KRBTEST.COM'} at 127.0.0.1:57902
I20260809 15:39:00.901242 20979 tablet_service.cc:1514] Processing DeleteTablet for tablet 1746f14a92d5425d935e609571748e70 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 15:39:00 UTC) from {username='kudu', principal='kudu/127.3.89.254@KRBTEST.COM'} at 127.0.0.1:46246
I20260809 15:39:00.901626 21086 tablet_replica.cc:330] T 1746f14a92d5425d935e609571748e70 P 7ba1b776d4734e909008ca7c997fde5f: stopping tablet replica
I20260809 15:39:00.901865 3431 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskm314YR/build/debug/bin/kudu with pid 20594
I20260809 15:39:00.901926 21086 raft_consensus.cc:2229] T 1746f14a92d5425d935e609571748e70 P 7ba1b776d4734e909008ca7c997fde5f [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 15:39:00.911580 21086 raft_consensus.cc:2258] T 1746f14a92d5425d935e609571748e70 P 7ba1b776d4734e909008ca7c997fde5f [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 15:39:00.912878 21086 ts_tablet_manager.cc:1905] T 1746f14a92d5425d935e609571748e70 P 7ba1b776d4734e909008ca7c997fde5f: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 15:39:00.915215 21087 tablet_replica.cc:330] T 1746f14a92d5425d935e609571748e70 P 440d47534a314ca88247d80185c766f8: stopping tablet replica
I20260809 15:39:00.915406 21086 ts_tablet_manager.cc:1918] T 1746f14a92d5425d935e609571748e70 P 7ba1b776d4734e909008ca7c997fde5f: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 15:39:00.915599 21086 log.cc:1196] T 1746f14a92d5425d935e609571748e70 P 7ba1b776d4734e909008ca7c997fde5f: Deleting WAL directory at /tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786289732823074-3431-0/minicluster-data/ts-1/wal/wals/1746f14a92d5425d935e609571748e70
I20260809 15:39:00.915762 21087 raft_consensus.cc:2229] T 1746f14a92d5425d935e609571748e70 P 440d47534a314ca88247d80185c766f8 [term 1 LEADER]: Raft consensus shutting down.
I20260809 15:39:00.916049 21086 ts_tablet_manager.cc:1939] T 1746f14a92d5425d935e609571748e70 P 7ba1b776d4734e909008ca7c997fde5f: Deleting consensus metadata
I20260809 15:39:00.916214 21087 raft_consensus.cc:2258] T 1746f14a92d5425d935e609571748e70 P 440d47534a314ca88247d80185c766f8 [term 1 FOLLOWER]: Raft consensus is shut down!
W20260809 15:39:00.916558 20532 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.3.89.193:36955 (error 108)
I20260809 15:39:00.917459 20532 catalog_manager.cc:4841] TS 7ba1b776d4734e909008ca7c997fde5f (127.3.89.194:32773): tablet 1746f14a92d5425d935e609571748e70 (table test-table [id=88f77c30fbe94b929b26dd067b365dde]) successfully deleted
I20260809 15:39:00.918149 3431 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskm314YR/build/debug/bin/kudu with pid 20764
I20260809 15:39:00.918876 21087 ts_tablet_manager.cc:1905] T 1746f14a92d5425d935e609571748e70 P 440d47534a314ca88247d80185c766f8: Deleting tablet data with delete state TABLET_DATA_DELETED
W20260809 15:39:00.921542 21061 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 15:39:00.917323 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.89.193:36955 (local address 127.0.0.1:42108)
0809 15:39:00.917485 (+ 162us) negotiation.cc:107] Waiting for socket to connect
0809 15:39:00.917503 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0809 15:39:00.917597 (+ 94us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 15:39:00.921451 (+ 3854us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.89.193:36955: BlockingRecv error: recv error from unknown peer: Transport endpoint is not connected (error 107)
Metrics: {"client-negotiator.queue_time_us":58}
I20260809 15:39:00.921943 21087 ts_tablet_manager.cc:1918] T 1746f14a92d5425d935e609571748e70 P 440d47534a314ca88247d80185c766f8: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
W20260809 15:39:00.921923 20532 catalog_manager.cc:4584] TS 23bd656811574169affd015915dd21e5 (127.3.89.193:36955): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 1746f14a92d5425d935e609571748e70: Network error: Client connection negotiation failed: client connection to 127.3.89.193:36955: BlockingRecv error: recv error from unknown peer: Transport endpoint is not connected (error 107)
I20260809 15:39:00.922065 21087 log.cc:1196] T 1746f14a92d5425d935e609571748e70 P 440d47534a314ca88247d80185c766f8: Deleting WAL directory at /tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786289732823074-3431-0/minicluster-data/ts-2/wal/wals/1746f14a92d5425d935e609571748e70
I20260809 15:39:00.922425 21087 ts_tablet_manager.cc:1939] T 1746f14a92d5425d935e609571748e70 P 440d47534a314ca88247d80185c766f8: Deleting consensus metadata
I20260809 15:39:00.923722 20531 catalog_manager.cc:4841] TS 440d47534a314ca88247d80185c766f8 (127.3.89.195:44003): tablet 1746f14a92d5425d935e609571748e70 (table test-table [id=88f77c30fbe94b929b26dd067b365dde]) successfully deleted
I20260809 15:39:00.928109 3431 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskm314YR/build/debug/bin/kudu with pid 20913
I20260809 15:39:00.935349 3431 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskm314YR/build/debug/bin/kudu with pid 20513
I20260809 15:39:20.943280 3431 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskm314YR/build/debug/bin/kudu
/tmp/dist-test-taskm314YR/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786289732823074-3431-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786289732823074-3431-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786289732823074-3431-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786289732823074-3431-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.3.89.254:41873
--webserver_interface=127.3.89.254
--webserver_port=35627
--keytab_file=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786289732823074-3431-0/krb5kdc/kudu_127.3.89.254.keytab
--principal=kudu/127.3.89.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.3.89.212:37503
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.3.89.254:41873
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20 with env {KRB5CCNAME=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786289732823074-3431-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786289732823074-3431-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786289732823074-3431-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 15:39:21.057549 21089 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 15:39:21.057778 21089 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 15:39:21.057853 21089 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 15:39:21.061813 21089 flags.cc:425] Enabled experimental flag: --authn_token_validity_seconds=20
W20260809 15:39:21.061859 21089 flags.cc:425] Enabled experimental flag: --authz_token_validity_seconds=20
W20260809 15:39:21.061880 21089 flags.cc:425] Enabled experimental flag: --tsk_rotation_seconds=20
W20260809 15:39:21.061894 21089 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 15:39:21.061911 21089 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 15:39:21.061929 21089 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 15:39:21.061945 21089 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 15:39:21.061962 21089 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 15:39:21.067215 21089 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.3.89.212:37503
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786289732823074-3431-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786289732823074-3431-0/minicluster-data/master-0/wal
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20
--ipki_ca_key_size=768
--master_addresses=127.3.89.254:41873
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786289732823074-3431-0/krb5kdc/kudu_127.3.89.254.keytab
--principal=kudu/127.3.89.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.3.89.254:41873
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786289732823074-3431-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.3.89.254
--webserver_port=35627
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.21089
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786289732823074-3431-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 09 Aug 2026 14:46:55 UTC on ccycloud.quasar-yswkmf.root.comops.site
build id 4576540
I20260809 15:39:21.068215 21089 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 15:39:21.069154 21089 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 15:39:21.077515 21095 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 15:39:21.077601 21089 server_base.cc:1034] running on GCE node
W20260809 15:39:21.077543 21094 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 15:39:21.077551 21097 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 15:39:21.078290 21089 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 15:39:21.079368 21089 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 15:39:21.080540 21089 hybrid_clock.cc:648] HybridClock initialized: now 1786289961080512 us; error 39 us; skew 500 ppm
Aug 09 15:39:21 dist-test-slave-2w3w krb5kdc[19918](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786289961, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.89.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 15:39:21.088006 21089 init.cc:377] Logged in from keytab as kudu/127.3.89.254@KRBTEST.COM (short username kudu)
I20260809 15:39:21.088886 21089 webserver.cc:460] Webserver started at http://127.3.89.254:35627/ using document root <none> and password file <none>
I20260809 15:39:21.089387 21089 fs_manager.cc:359] Metadata directory not provided
I20260809 15:39:21.089447 21089 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 15:39:21.093287 21089 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.004s sys 0.000s
I20260809 15:39:21.095342 21105 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 15:39:21.096515 21089 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260809 15:39:21.096640 21089 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786289732823074-3431-0/minicluster-data/master-0/data,/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786289732823074-3431-0/minicluster-data/master-0/wal
uuid: "210f7703a6ee4761acec0985806eb015"
format_stamp: "Formatted at 2026-08-09 15:38:38 on dist-test-slave-2w3w"
server_key: "03ecf0ac536692172efc79bd865fd7d4"
server_key_iv: "397c712c73463366eb9a3a049bad8e83"
server_key_version: "encryptionkey@0"
I20260809 15:39:21.097172 21089 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786289732823074-3431-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786289732823074-3431-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786289732823074-3431-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 15:39:21.109975 21089 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 15:39:21.112703 21089 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 15:39:21.112903 21089 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 15:39:21.119896 21089 rpc_server.cc:307] RPC server started. Bound to: 127.3.89.254:41873
I20260809 15:39:21.120177 21159 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.3.89.254:41873 every 8 connection(s)
I20260809 15:39:21.120922 21089 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786289732823074-3431-0/minicluster-data/master-0/data/info.pb
I20260809 15:39:21.124943 21160 sys_catalog.cc:263] Verifying existing consensus state
I20260809 15:39:21.126432 21160 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 210f7703a6ee4761acec0985806eb015: Bootstrap starting.
I20260809 15:39:21.130865 3431 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskm314YR/build/debug/bin/kudu as pid 21089
I20260809 15:39:21.131417 3431 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskm314YR/build/debug/bin/kudu
/tmp/dist-test-taskm314YR/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786289732823074-3431-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786289732823074-3431-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786289732823074-3431-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786289732823074-3431-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.3.89.193:36955
--local_ip_for_outbound_sockets=127.3.89.193
--tserver_master_addrs=127.3.89.254:41873
--webserver_port=44001
--webserver_interface=127.3.89.193
--keytab_file=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786289732823074-3431-0/krb5kdc/kudu_127.3.89.193.keytab
--principal=kudu/127.3.89.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.3.89.212:37503
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786289732823074-3431-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786289732823074-3431-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786289732823074-3431-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260809 15:39:21.145013 21160 log.cc:824] T 00000000000000000000000000000000 P 210f7703a6ee4761acec0985806eb015: Log is configured to *not* fsync() on all Append() calls
I20260809 15:39:21.155937 21160 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 210f7703a6ee4761acec0985806eb015: Bootstrap replayed 1/1 log segments. Stats: ops{read=14 overwritten=0 applied=14 ignored=0} inserts{seen=8 ignored=0} mutations{seen=8 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260809 15:39:21.156342 21160 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 210f7703a6ee4761acec0985806eb015: Bootstrap complete.
I20260809 15:39:21.160310 21160 raft_consensus.cc:348] T 00000000000000000000000000000000 P 210f7703a6ee4761acec0985806eb015 [term 2 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "210f7703a6ee4761acec0985806eb015" member_type: VOTER last_known_addr { host: "127.3.89.254" port: 41873 } }
I20260809 15:39:21.160671 21160 raft_consensus.cc:729] T 00000000000000000000000000000000 P 210f7703a6ee4761acec0985806eb015 [term 2 FOLLOWER]: Becoming Follower/Learner. State: Replica: 210f7703a6ee4761acec0985806eb015, State: Initialized, Role: FOLLOWER
I20260809 15:39:21.161177 21160 consensus_queue.cc:260] T 00000000000000000000000000000000 P 210f7703a6ee4761acec0985806eb015 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 14, Last appended: 2.14, Last appended by leader: 14, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "210f7703a6ee4761acec0985806eb015" member_type: VOTER last_known_addr { host: "127.3.89.254" port: 41873 } }
I20260809 15:39:21.161293 21160 raft_consensus.cc:388] T 00000000000000000000000000000000 P 210f7703a6ee4761acec0985806eb015 [term 2 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 15:39:21.161365 21160 raft_consensus.cc:482] T 00000000000000000000000000000000 P 210f7703a6ee4761acec0985806eb015 [term 2 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 15:39:21.161486 21160 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 210f7703a6ee4761acec0985806eb015 [term 2 FOLLOWER]: Advancing to term 3
I20260809 15:39:21.163394 21160 raft_consensus.cc:504] T 00000000000000000000000000000000 P 210f7703a6ee4761acec0985806eb015 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "210f7703a6ee4761acec0985806eb015" member_type: VOTER last_known_addr { host: "127.3.89.254" port: 41873 } }
I20260809 15:39:21.163708 21160 leader_election.cc:302] T 00000000000000000000000000000000 P 210f7703a6ee4761acec0985806eb015 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 210f7703a6ee4761acec0985806eb015; no voters:
I20260809 15:39:21.164057 21160 leader_election.cc:288] T 00000000000000000000000000000000 P 210f7703a6ee4761acec0985806eb015 [CANDIDATE]: Term 3 election: Requested vote from peers
I20260809 15:39:21.164157 21164 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 210f7703a6ee4761acec0985806eb015 [term 3 FOLLOWER]: Leader election won for term 3
I20260809 15:39:21.164427 21164 raft_consensus.cc:686] T 00000000000000000000000000000000 P 210f7703a6ee4761acec0985806eb015 [term 3 LEADER]: Becoming Leader. State: Replica: 210f7703a6ee4761acec0985806eb015, State: Running, Role: LEADER
I20260809 15:39:21.164791 21164 consensus_queue.cc:237] T 00000000000000000000000000000000 P 210f7703a6ee4761acec0985806eb015 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 2.14, Last appended by leader: 14, Current term: 3, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "210f7703a6ee4761acec0985806eb015" member_type: VOTER last_known_addr { host: "127.3.89.254" port: 41873 } }
I20260809 15:39:21.165236 21160 sys_catalog.cc:564] T 00000000000000000000000000000000 P 210f7703a6ee4761acec0985806eb015 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 15:39:21.166103 21165 sys_catalog.cc:455] T 00000000000000000000000000000000 P 210f7703a6ee4761acec0985806eb015 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 3 leader_uuid: "210f7703a6ee4761acec0985806eb015" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "210f7703a6ee4761acec0985806eb015" member_type: VOTER last_known_addr { host: "127.3.89.254" port: 41873 } } }
I20260809 15:39:21.166322 21165 sys_catalog.cc:458] T 00000000000000000000000000000000 P 210f7703a6ee4761acec0985806eb015 [sys.catalog]: This master's current role is: LEADER
I20260809 15:39:21.166604 21165 sys_catalog.cc:455] T 00000000000000000000000000000000 P 210f7703a6ee4761acec0985806eb015 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 210f7703a6ee4761acec0985806eb015. Latest consensus state: current_term: 3 leader_uuid: "210f7703a6ee4761acec0985806eb015" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "210f7703a6ee4761acec0985806eb015" member_type: VOTER last_known_addr { host: "127.3.89.254" port: 41873 } } }
I20260809 15:39:21.167209 21165 sys_catalog.cc:458] T 00000000000000000000000000000000 P 210f7703a6ee4761acec0985806eb015 [sys.catalog]: This master's current role is: LEADER
I20260809 15:39:21.170043 21183 catalog_manager.cc:1259] Loaded cluster ID: bc02bf064fd44d40b13be3238ff5b7dc
I20260809 15:39:21.170100 21183 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 210f7703a6ee4761acec0985806eb015: loading cluster ID for follower catalog manager: success
I20260809 15:39:21.172509 21183 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 210f7703a6ee4761acec0985806eb015: acquiring CA information for follower catalog manager: success
F20260809 15:39:21.173686 21183 token_signing_key.cc:66] Check failed: _s.ok() Bad status: Runtime error: unable to load data from memory: error:0D0680A8:asn1 encoding routines:ASN1_CHECK_TLEN:wrong tag:tasn_dec.c:1220 error:0D06C03A:asn1 encoding routines:ASN1_D2I_EX_PRIMITIVE:nested asn1 error:tasn_dec.c:788 error:0D08303A:asn1 encoding routines:ASN1_TEMPLATE_NOEXP_D2I:nested asn1 error:tasn_dec.c:720:Field=version, Type=RSA error:04093004:rsa routines:OLD_RSA_PRIV_DECODE:RSA lib:rsa_ameth.c:121 error:0D0680A8:asn1 encoding routines:ASN1_CHECK_TLEN:wrong tag:tasn_dec.c:1220 error:0D06C03A:asn1 encoding routines:ASN1_D2I_EX_PRIMITIVE:nested asn1 error:tasn_dec.c:788 error:0D08303A:asn1 encoding routines:ASN1_TEMPLATE_NOEXP_D2I:nested asn1 error:tasn_dec.c:720:Field=version, Type=PKCS8_PRIV_KEY_INFO
*** Check failure stack trace: ***
@ 0x7f3f02e399ad google::LogMessage::Fail() at ??:0
@ 0x7f3f02e3c88a google::LogMessage::SendToLog() at ??:0
@ 0x7f3f02e394de google::LogMessage::Flush() at ??:0
@ 0x7f3f02e3aa19 google::LogMessageFatal::~LogMessageFatal() at ??:0
@ 0x7f3f05afd18d kudu::security::TokenSigningPrivateKey::TokenSigningPrivateKey() at ??:0
@ 0x7f3f0d7a2989 kudu::master::CatalogManager::LoadTspkEntries() at ??:0
@ 0x7f3f0d77f023 kudu::master::CatalogManager::PrepareFollowerTokenVerifier() at ??:0
@ 0x7f3f0d77f54b kudu::master::CatalogManager::PrepareFollower() at ??:0
@ 0x7f3f0d778652 kudu::master::CatalogManagerBgTasks::Run() at ??:0
@ 0x7f3f0d7779e9 _ZZN4kudu6master21CatalogManagerBgTasks4InitEvENKUlvE_clEv at ??:0
@ 0x7f3f0d7be541 _ZNSt17_Function_handlerIFvvEZN4kudu6master21CatalogManagerBgTasks4InitEvEUlvE_E9_M_invokeERKSt9_Any_data at ??:0
@ 0x61d838 std::function<>::operator()() at ??:0
@ 0x7f3f0507c512 kudu::Thread::SuperviseThread() at ??:0
@ 0x7f3f066c06db start_thread at ??:0
@ 0x7f3f0193761f clone at ??:0
W20260809 15:39:21.259069 21162 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 15:39:21.259325 21162 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 15:39:21.259419 21162 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 15:39:21.263367 21162 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 15:39:21.263438 21162 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 15:39:21.263541 21162 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.3.89.193
I20260809 15:39:21.268790 21162 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.3.89.212:37503
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786289732823074-3431-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786289732823074-3431-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786289732823074-3431-0/krb5kdc/kudu_127.3.89.193.keytab
--principal=kudu/127.3.89.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.3.89.193:36955
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786289732823074-3431-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.3.89.193
--webserver_port=44001
--tserver_master_addrs=127.3.89.254:41873
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.21162
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.3.89.193
--log_dir=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786289732823074-3431-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 09 Aug 2026 14:46:55 UTC on ccycloud.quasar-yswkmf.root.comops.site
build id 4576540
I20260809 15:39:21.269791 21162 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 15:39:21.270730 21162 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 15:39:21.279850 21204 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 15:39:21.279831 21207 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 15:39:21.279974 21205 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 15:39:21.280638 21162 server_base.cc:1034] running on GCE node
I20260809 15:39:21.281029 21162 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 15:39:21.281637 21162 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 15:39:21.282837 21162 hybrid_clock.cc:648] HybridClock initialized: now 1786289961282814 us; error 44 us; skew 500 ppm
Aug 09 15:39:21 dist-test-slave-2w3w krb5kdc[19918](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786289961, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.89.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 15:39:21.285526 21162 init.cc:377] Logged in from keytab as kudu/127.3.89.193@KRBTEST.COM (short username kudu)
I20260809 15:39:21.286387 21162 webserver.cc:460] Webserver started at http://127.3.89.193:44001/ using document root <none> and password file <none>
I20260809 15:39:21.286904 21162 fs_manager.cc:359] Metadata directory not provided
I20260809 15:39:21.287009 21162 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 15:39:21.290670 21162 fs_manager.cc:709] Time spent opening directory manager: real 0.002s user 0.003s sys 0.000s
I20260809 15:39:21.292922 21214 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 15:39:21.294313 21162 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.000s sys 0.002s
I20260809 15:39:21.294510 21162 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786289732823074-3431-0/minicluster-data/ts-0/data,/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786289732823074-3431-0/minicluster-data/ts-0/wal
uuid: "23bd656811574169affd015915dd21e5"
format_stamp: "Formatted at 2026-08-09 15:38:38 on dist-test-slave-2w3w"
server_key: "a4aab3d472b4637e64f780d2befe66ff"
server_key_iv: "8968b27898f895919b075309b448b735"
server_key_version: "encryptionkey@0"
I20260809 15:39:21.295234 21162 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786289732823074-3431-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786289732823074-3431-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786289732823074-3431-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 15:39:21.314219 21162 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 15:39:21.317436 21162 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 15:39:21.317647 21162 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 15:39:21.318295 21162 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 15:39:21.319490 21222 ts_tablet_manager.cc:536] Loading tablet metadata (0/1 complete)
I20260809 15:39:21.323508 21162 ts_tablet_manager.cc:579] Loaded tablet metadata (1 total tablets, 1 live tablets)
I20260809 15:39:21.323570 21162 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.004s user 0.000s sys 0.000s
I20260809 15:39:21.323638 21162 ts_tablet_manager.cc:594] Registering tablets (0/1 complete)
I20260809 15:39:21.325241 21162 ts_tablet_manager.cc:610] Registered 1 tablets
I20260809 15:39:21.325299 21162 ts_tablet_manager.cc:589] Time spent register tablets: real 0.002s user 0.000s sys 0.000s
I20260809 15:39:21.325552 21222 tablet_bootstrap.cc:492] T 1746f14a92d5425d935e609571748e70 P 23bd656811574169affd015915dd21e5: Bootstrap starting.
I20260809 15:39:21.336369 21162 rpc_server.cc:307] RPC server started. Bound to: 127.3.89.193:36955
I20260809 15:39:21.336616 21340 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.3.89.193:36955 every 8 connection(s)
I20260809 15:39:21.337549 21162 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786289732823074-3431-0/minicluster-data/ts-0/data/info.pb
I20260809 15:39:21.339008 3431 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskm314YR/build/debug/bin/kudu as pid 21162
I20260809 15:39:21.339560 3431 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskm314YR/build/debug/bin/kudu
/tmp/dist-test-taskm314YR/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786289732823074-3431-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786289732823074-3431-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786289732823074-3431-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786289732823074-3431-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.3.89.194:32773
--local_ip_for_outbound_sockets=127.3.89.194
--tserver_master_addrs=127.3.89.254:41873
--webserver_port=39463
--webserver_interface=127.3.89.194
--keytab_file=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786289732823074-3431-0/krb5kdc/kudu_127.3.89.194.keytab
--principal=kudu/127.3.89.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.3.89.212:37503
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786289732823074-3431-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786289732823074-3431-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786289732823074-3431-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260809 15:39:21.346678 21222 log.cc:824] T 1746f14a92d5425d935e609571748e70 P 23bd656811574169affd015915dd21e5: Log is configured to *not* fsync() on all Append() calls
I20260809 15:39:21.353217 21222 tablet_bootstrap.cc:492] T 1746f14a92d5425d935e609571748e70 P 23bd656811574169affd015915dd21e5: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=1 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260809 15:39:21.353591 21222 tablet_bootstrap.cc:492] T 1746f14a92d5425d935e609571748e70 P 23bd656811574169affd015915dd21e5: Bootstrap complete.
I20260809 15:39:21.354694 21222 ts_tablet_manager.cc:1397] T 1746f14a92d5425d935e609571748e70 P 23bd656811574169affd015915dd21e5: Time spent bootstrapping tablet: real 0.029s user 0.011s sys 0.015s
I20260809 15:39:21.356984 21222 raft_consensus.cc:348] T 1746f14a92d5425d935e609571748e70 P 23bd656811574169affd015915dd21e5 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7ba1b776d4734e909008ca7c997fde5f" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 32773 } } peers { permanent_uuid: "440d47534a314ca88247d80185c766f8" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 44003 } } peers { permanent_uuid: "23bd656811574169affd015915dd21e5" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 36955 } }
I20260809 15:39:21.357422 21222 raft_consensus.cc:729] T 1746f14a92d5425d935e609571748e70 P 23bd656811574169affd015915dd21e5 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 23bd656811574169affd015915dd21e5, State: Initialized, Role: FOLLOWER
I20260809 15:39:21.357880 21222 consensus_queue.cc:260] T 1746f14a92d5425d935e609571748e70 P 23bd656811574169affd015915dd21e5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7ba1b776d4734e909008ca7c997fde5f" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 32773 } } peers { permanent_uuid: "440d47534a314ca88247d80185c766f8" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 44003 } } peers { permanent_uuid: "23bd656811574169affd015915dd21e5" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 36955 } }
I20260809 15:39:21.358764 21222 ts_tablet_manager.cc:1428] T 1746f14a92d5425d935e609571748e70 P 23bd656811574169affd015915dd21e5: Time spent starting tablet: real 0.004s user 0.006s sys 0.002s
W20260809 15:39:21.397505 21343 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 15:39:21.339698 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.89.254:41873 (local address 127.3.89.193:44347)
0809 15:39:21.341090 (+ 1392us) negotiation.cc:107] Waiting for socket to connect
0809 15:39:21.341149 (+ 59us) client_negotiation.cc:174] Beginning negotiation
0809 15:39:21.342238 (+ 1089us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 15:39:21.396776 (+ 54538us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.89.254:41873: BlockingRecv error: recv error from unknown peer: Transport endpoint is not connected (error 107)
Metrics: {"client-negotiator.queue_time_us":1155,"thread_start_us":135,"threads_started":1}
W20260809 15:39:21.397997 21341 heartbeater.cc:644] Failed to heartbeat to 127.3.89.254:41873 (0 consecutive failures): Network error: Failed to ping master at 127.3.89.254:41873: Client connection negotiation failed: client connection to 127.3.89.254:41873: BlockingRecv error: recv error from unknown peer: Transport endpoint is not connected (error 107)
I20260809 15:39:21.398842 21343 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:39:21.398519 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.89.254:41873 (local address 127.3.89.193:46413)
0809 15:39:21.398641 (+ 122us) negotiation.cc:107] Waiting for socket to connect
0809 15:39:21.398701 (+ 60us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.89.254:41873: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":48}
I20260809 15:39:21.399559 21343 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:39:21.399342 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.89.254:41873 (local address 127.3.89.193:36515)
0809 15:39:21.399452 (+ 110us) negotiation.cc:107] Waiting for socket to connect
0809 15:39:21.399502 (+ 50us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.89.254:41873: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":49}
W20260809 15:39:21.399879 21341 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260809 15:39:21.470036 21344 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 15:39:21.470366 21344 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 15:39:21.470451 21344 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 15:39:21.474794 21344 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 15:39:21.474851 21344 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 15:39:21.474953 21344 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.3.89.194
I20260809 15:39:21.480020 21344 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.3.89.212:37503
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786289732823074-3431-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786289732823074-3431-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786289732823074-3431-0/krb5kdc/kudu_127.3.89.194.keytab
--principal=kudu/127.3.89.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.3.89.194:32773
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786289732823074-3431-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.3.89.194
--webserver_port=39463
--tserver_master_addrs=127.3.89.254:41873
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.21344
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.3.89.194
--log_dir=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786289732823074-3431-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 09 Aug 2026 14:46:55 UTC on ccycloud.quasar-yswkmf.root.comops.site
build id 4576540
I20260809 15:39:21.480995 21344 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 15:39:21.481918 21344 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 15:39:21.491195 21351 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 15:39:21.491232 21344 server_base.cc:1034] running on GCE node
W20260809 15:39:21.491190 21354 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 15:39:21.491190 21352 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 15:39:21.491811 21344 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 15:39:21.492494 21344 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 15:39:21.493667 21344 hybrid_clock.cc:648] HybridClock initialized: now 1786289961493638 us; error 41 us; skew 500 ppm
Aug 09 15:39:21 dist-test-slave-2w3w krb5kdc[19918](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786289961, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.89.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 15:39:21.496245 21344 init.cc:377] Logged in from keytab as kudu/127.3.89.194@KRBTEST.COM (short username kudu)
I20260809 15:39:21.497027 21344 webserver.cc:460] Webserver started at http://127.3.89.194:39463/ using document root <none> and password file <none>
I20260809 15:39:21.497560 21344 fs_manager.cc:359] Metadata directory not provided
I20260809 15:39:21.497653 21344 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 15:39:21.501420 21344 fs_manager.cc:709] Time spent opening directory manager: real 0.002s user 0.001s sys 0.000s
I20260809 15:39:21.503362 21361 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 15:39:21.504499 21344 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260809 15:39:21.504637 21344 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786289732823074-3431-0/minicluster-data/ts-1/data,/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786289732823074-3431-0/minicluster-data/ts-1/wal
uuid: "7ba1b776d4734e909008ca7c997fde5f"
format_stamp: "Formatted at 2026-08-09 15:38:39 on dist-test-slave-2w3w"
server_key: "05ddc3e034248186ff04a1599e9b0be6"
server_key_iv: "ac5b212a1983e3e9d8ae56c81442b5d1"
server_key_version: "encryptionkey@0"
I20260809 15:39:21.505147 21344 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786289732823074-3431-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786289732823074-3431-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786289732823074-3431-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 15:39:21.528297 21344 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 15:39:21.531098 21344 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 15:39:21.531311 21344 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 15:39:21.531936 21344 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 15:39:21.532773 21344 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 15:39:21.532850 21344 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 15:39:21.532919 21344 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 15:39:21.532963 21344 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 15:39:21.541630 21344 rpc_server.cc:307] RPC server started. Bound to: 127.3.89.194:32773
I20260809 15:39:21.541721 21485 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.3.89.194:32773 every 8 connection(s)
I20260809 15:39:21.542671 21344 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786289732823074-3431-0/minicluster-data/ts-1/data/info.pb
I20260809 15:39:21.546344 21488 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:39:21.544633 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.89.254:41873 (local address 127.3.89.194:53813)
0809 15:39:21.545067 (+ 434us) negotiation.cc:107] Waiting for socket to connect
0809 15:39:21.545183 (+ 116us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.89.254:41873: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":243,"thread_start_us":113,"threads_started":1}
W20260809 15:39:21.546814 21486 heartbeater.cc:644] Failed to heartbeat to 127.3.89.254:41873 (0 consecutive failures): Network error: Failed to ping master at 127.3.89.254:41873: Client connection negotiation failed: client connection to 127.3.89.254:41873: connect: Connection refused (error 111)
I20260809 15:39:21.547397 21488 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:39:21.547198 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.89.254:41873 (local address 127.3.89.194:33767)
0809 15:39:21.547304 (+ 106us) negotiation.cc:107] Waiting for socket to connect
0809 15:39:21.547348 (+ 44us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.89.254:41873: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":37}
I20260809 15:39:21.547938 21488 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:39:21.547769 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.89.254:41873 (local address 127.3.89.194:60601)
0809 15:39:21.547858 (+ 89us) negotiation.cc:107] Waiting for socket to connect
0809 15:39:21.547896 (+ 38us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.89.254:41873: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":37}
W20260809 15:39:21.548115 21486 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
I20260809 15:39:21.551982 3431 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskm314YR/build/debug/bin/kudu as pid 21344
I20260809 15:39:21.552392 3431 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskm314YR/build/debug/bin/kudu
/tmp/dist-test-taskm314YR/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786289732823074-3431-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786289732823074-3431-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786289732823074-3431-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786289732823074-3431-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.3.89.195:44003
--local_ip_for_outbound_sockets=127.3.89.195
--tserver_master_addrs=127.3.89.254:41873
--webserver_port=43913
--webserver_interface=127.3.89.195
--keytab_file=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786289732823074-3431-0/krb5kdc/kudu_127.3.89.195.keytab
--principal=kudu/127.3.89.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.3.89.212:37503
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786289732823074-3431-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786289732823074-3431-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786289732823074-3431-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 15:39:21.661921 21489 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 15:39:21.662159 21489 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 15:39:21.662200 21489 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 15:39:21.666220 21489 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 15:39:21.666283 21489 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 15:39:21.666339 21489 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.3.89.195
I20260809 15:39:21.671478 21489 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.3.89.212:37503
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786289732823074-3431-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786289732823074-3431-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786289732823074-3431-0/krb5kdc/kudu_127.3.89.195.keytab
--principal=kudu/127.3.89.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.3.89.195:44003
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786289732823074-3431-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.3.89.195
--webserver_port=43913
--tserver_master_addrs=127.3.89.254:41873
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.21489
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.3.89.195
--log_dir=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786289732823074-3431-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 09 Aug 2026 14:46:55 UTC on ccycloud.quasar-yswkmf.root.comops.site
build id 4576540
I20260809 15:39:21.672456 21489 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 15:39:21.673398 21489 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 15:39:21.682812 21497 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 15:39:21.682802 21495 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 15:39:21.682797 21494 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 15:39:21.683182 21489 server_base.cc:1034] running on GCE node
I20260809 15:39:21.683604 21489 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 15:39:21.684255 21489 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 15:39:21.685454 21489 hybrid_clock.cc:648] HybridClock initialized: now 1786289961685433 us; error 34 us; skew 500 ppm
Aug 09 15:39:21 dist-test-slave-2w3w krb5kdc[19918](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786289961, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.89.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 15:39:21.688310 21489 init.cc:377] Logged in from keytab as kudu/127.3.89.195@KRBTEST.COM (short username kudu)
I20260809 15:39:21.689181 21489 webserver.cc:460] Webserver started at http://127.3.89.195:43913/ using document root <none> and password file <none>
I20260809 15:39:21.689752 21489 fs_manager.cc:359] Metadata directory not provided
I20260809 15:39:21.689848 21489 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 15:39:21.693710 21489 fs_manager.cc:709] Time spent opening directory manager: real 0.002s user 0.000s sys 0.001s
I20260809 15:39:21.695755 21504 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 15:39:21.696985 21489 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260809 15:39:21.697127 21489 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786289732823074-3431-0/minicluster-data/ts-2/data,/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786289732823074-3431-0/minicluster-data/ts-2/wal
uuid: "440d47534a314ca88247d80185c766f8"
format_stamp: "Formatted at 2026-08-09 15:38:39 on dist-test-slave-2w3w"
server_key: "62d7b19b3c4c4511cb8d34b326b0568e"
server_key_iv: "1f6e4f692caf051f3d781a8ccd66e70b"
server_key_version: "encryptionkey@0"
I20260809 15:39:21.697687 21489 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786289732823074-3431-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786289732823074-3431-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786289732823074-3431-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 15:39:21.710831 21489 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 15:39:21.713640 21489 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 15:39:21.713851 21489 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 15:39:21.714437 21489 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 15:39:21.715343 21489 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 15:39:21.715413 21489 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 15:39:21.715479 21489 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 15:39:21.715528 21489 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 15:39:21.724692 21489 rpc_server.cc:307] RPC server started. Bound to: 127.3.89.195:44003
I20260809 15:39:21.724715 21631 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.3.89.195:44003 every 8 connection(s)
I20260809 15:39:21.725646 21489 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskm314YR/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786289732823074-3431-0/minicluster-data/ts-2/data/info.pb
I20260809 15:39:21.729067 3431 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskm314YR/build/debug/bin/kudu as pid 21489
I20260809 15:39:21.729135 21634 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:39:21.727707 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.89.254:41873 (local address 127.3.89.195:58619)
0809 15:39:21.728363 (+ 656us) negotiation.cc:107] Waiting for socket to connect
0809 15:39:21.728470 (+ 107us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.89.254:41873: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":464,"thread_start_us":115,"threads_started":1}
W20260809 15:39:21.729638 21632 heartbeater.cc:644] Failed to heartbeat to 127.3.89.254:41873 (0 consecutive failures): Network error: Failed to ping master at 127.3.89.254:41873: Client connection negotiation failed: client connection to 127.3.89.254:41873: connect: Connection refused (error 111)
I20260809 15:39:21.730257 21634 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:39:21.730015 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.89.254:41873 (local address 127.3.89.195:52897)
0809 15:39:21.730134 (+ 119us) negotiation.cc:107] Waiting for socket to connect
0809 15:39:21.730185 (+ 51us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.89.254:41873: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":48}
I20260809 15:39:21.730844 21634 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:39:21.730621 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.89.254:41873 (local address 127.3.89.195:37165)
0809 15:39:21.730716 (+ 95us) negotiation.cc:107] Waiting for socket to connect
0809 15:39:21.730793 (+ 77us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.89.254:41873: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":39}
W20260809 15:39:21.731030 21632 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
I20260809 15:39:22.400846 21636 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:39:22.400385 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.89.254:41873 (local address 127.3.89.193:34887)
0809 15:39:22.400666 (+ 281us) negotiation.cc:107] Waiting for socket to connect
0809 15:39:22.400719 (+ 53us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.89.254:41873: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":205,"thread_start_us":80,"threads_started":1}
I20260809 15:39:22.548986 21637 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:39:22.548623 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.89.254:41873 (local address 127.3.89.194:53475)
0809 15:39:22.548845 (+ 222us) negotiation.cc:107] Waiting for socket to connect
0809 15:39:22.548894 (+ 49us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.89.254:41873: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":160,"thread_start_us":69,"threads_started":1}
I20260809 15:39:22.660818 21638 raft_consensus.cc:482] T 1746f14a92d5425d935e609571748e70 P 23bd656811574169affd015915dd21e5 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 15:39:22.661029 21638 raft_consensus.cc:504] T 1746f14a92d5425d935e609571748e70 P 23bd656811574169affd015915dd21e5 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7ba1b776d4734e909008ca7c997fde5f" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 32773 } } peers { permanent_uuid: "440d47534a314ca88247d80185c766f8" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 44003 } } peers { permanent_uuid: "23bd656811574169affd015915dd21e5" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 36955 } }
I20260809 15:39:22.662114 21638 leader_election.cc:288] T 1746f14a92d5425d935e609571748e70 P 23bd656811574169affd015915dd21e5 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 7ba1b776d4734e909008ca7c997fde5f (127.3.89.194:32773), 440d47534a314ca88247d80185c766f8 (127.3.89.195:44003)
Aug 09 15:39:22 dist-test-slave-2w3w krb5kdc[19918](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786289961, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.89.193@KRBTEST.COM for kudu/127.3.89.194@KRBTEST.COM
Aug 09 15:39:22 dist-test-slave-2w3w krb5kdc[19918](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786289961, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.89.193@KRBTEST.COM for kudu/127.3.89.195@KRBTEST.COM
I20260809 15:39:22.676810 21636 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:39:22.662237 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.89.194:32773 (local address 127.3.89.193:49575)
0809 15:39:22.662387 (+ 150us) negotiation.cc:107] Waiting for socket to connect
0809 15:39:22.662403 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0809 15:39:22.662511 (+ 108us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 15:39:22.664444 (+ 1933us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 15:39:22.664450 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 15:39:22.664975 (+ 525us) client_negotiation.cc:189] Negotiated authn=SASL
0809 15:39:22.665450 (+ 475us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:39:22.665464 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:39:22.667060 (+ 1596us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:39:22.667062 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:39:22.668024 (+ 962us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:39:22.668034 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:39:22.668750 (+ 716us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:39:22.668754 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:39:22.668975 (+ 221us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:39:22.669636 (+ 661us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 15:39:22.669652 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0809 15:39:22.671364 (+ 1712us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 15:39:22.673833 (+ 2469us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 15:39:22.673838 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 15:39:22.673848 (+ 10us) client_negotiation.cc:760] calling sasl_client_step()
0809 15:39:22.674235 (+ 387us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 15:39:22.674625 (+ 390us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 15:39:22.674627 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 15:39:22.674629 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 15:39:22.674824 (+ 195us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 15:39:22.675606 (+ 782us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 15:39:22.675612 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 15:39:22.675991 (+ 379us) client_negotiation.cc:769] Sending connection context
0809 15:39:22.676207 (+ 216us) client_negotiation.cc:240] Negotiation successful
0809 15:39:22.676593 (+ 386us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":66,"mutex_wait_us":251,"spinlock_wait_cycles":27904}
I20260809 15:39:22.676833 21640 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:39:22.662667 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.89.193:49575 (local address 127.3.89.194:32773)
0809 15:39:22.663053 (+ 386us) server_negotiation.cc:206] Beginning negotiation
0809 15:39:22.663074 (+ 21us) server_negotiation.cc:399] Waiting for connection header
0809 15:39:22.663114 (+ 40us) server_negotiation.cc:407] Connection header received
0809 15:39:22.663790 (+ 676us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 15:39:22.663795 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 15:39:22.664098 (+ 303us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 15:39:22.664361 (+ 263us) server_negotiation.cc:226] Negotiated authn=SASL
0809 15:39:22.665633 (+ 1272us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:39:22.666870 (+ 1237us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:39:22.668155 (+ 1285us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:39:22.668619 (+ 464us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:39:22.668801 (+ 182us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:39:22.671595 (+ 2794us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 15:39:22.671600 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 15:39:22.671611 (+ 11us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 15:39:22.671639 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0809 15:39:22.673671 (+ 2032us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 15:39:22.674351 (+ 680us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 15:39:22.674355 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 15:39:22.674359 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 15:39:22.674526 (+ 167us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 15:39:22.674942 (+ 416us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 15:39:22.674945 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 15:39:22.674947 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 15:39:22.675468 (+ 521us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 15:39:22.675610 (+ 142us) server_negotiation.cc:1035] Waiting for connection context
0809 15:39:22.676354 (+ 744us) server_negotiation.cc:299] Negotiation successful
0809 15:39:22.676645 (+ 291us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":190,"thread_start_us":77,"threads_started":1}
I20260809 15:39:22.677059 21641 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:39:22.662630 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.89.193:46123 (local address 127.3.89.195:44003)
0809 15:39:22.663133 (+ 503us) server_negotiation.cc:206] Beginning negotiation
0809 15:39:22.663144 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0809 15:39:22.663176 (+ 32us) server_negotiation.cc:407] Connection header received
0809 15:39:22.663764 (+ 588us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 15:39:22.663769 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 15:39:22.664097 (+ 328us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 15:39:22.664399 (+ 302us) server_negotiation.cc:226] Negotiated authn=SASL
0809 15:39:22.665629 (+ 1230us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:39:22.666836 (+ 1207us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:39:22.668153 (+ 1317us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:39:22.668618 (+ 465us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:39:22.668802 (+ 184us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:39:22.671957 (+ 3155us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 15:39:22.671964 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 15:39:22.671979 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 15:39:22.672014 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0809 15:39:22.673933 (+ 1919us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 15:39:22.674488 (+ 555us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 15:39:22.674501 (+ 13us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 15:39:22.674506 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0809 15:39:22.674664 (+ 158us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 15:39:22.674942 (+ 278us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 15:39:22.674945 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 15:39:22.674947 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 15:39:22.675484 (+ 537us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 15:39:22.675610 (+ 126us) server_negotiation.cc:1035] Waiting for connection context
0809 15:39:22.676354 (+ 744us) server_negotiation.cc:299] Negotiation successful
0809 15:39:22.676889 (+ 535us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":402,"thread_start_us":132,"threads_started":1}
I20260809 15:39:22.676810 21639 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:39:22.662399 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.89.195:44003 (local address 127.3.89.193:46123)
0809 15:39:22.662770 (+ 371us) negotiation.cc:107] Waiting for socket to connect
0809 15:39:22.662783 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0809 15:39:22.662870 (+ 87us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 15:39:22.664735 (+ 1865us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 15:39:22.664737 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 15:39:22.664978 (+ 241us) client_negotiation.cc:189] Negotiated authn=SASL
0809 15:39:22.665450 (+ 472us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:39:22.665471 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:39:22.667003 (+ 1532us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:39:22.667008 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:39:22.668025 (+ 1017us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:39:22.668035 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:39:22.668748 (+ 713us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:39:22.668752 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:39:22.668968 (+ 216us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:39:22.669690 (+ 722us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 15:39:22.669702 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0809 15:39:22.671747 (+ 2045us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 15:39:22.674042 (+ 2295us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 15:39:22.674047 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 15:39:22.674050 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0809 15:39:22.674371 (+ 321us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 15:39:22.674765 (+ 394us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 15:39:22.674768 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 15:39:22.674770 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 15:39:22.674849 (+ 79us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 15:39:22.675606 (+ 757us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 15:39:22.675612 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 15:39:22.675991 (+ 379us) client_negotiation.cc:769] Sending connection context
0809 15:39:22.676207 (+ 216us) client_negotiation.cc:240] Negotiation successful
0809 15:39:22.676646 (+ 439us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":296,"mutex_wait_us":264,"spinlock_wait_cycles":30976,"thread_start_us":113,"threads_started":1}
W20260809 15:39:22.680270 21216 leader_election.cc:341] T 1746f14a92d5425d935e609571748e70 P 23bd656811574169affd015915dd21e5 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 7ba1b776d4734e909008ca7c997fde5f (127.3.89.194:32773): Not found: Tablet not found: 1746f14a92d5425d935e609571748e70
W20260809 15:39:22.680459 21215 leader_election.cc:341] T 1746f14a92d5425d935e609571748e70 P 23bd656811574169affd015915dd21e5 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 440d47534a314ca88247d80185c766f8 (127.3.89.195:44003): Not found: Tablet not found: 1746f14a92d5425d935e609571748e70
I20260809 15:39:22.680544 21215 leader_election.cc:302] T 1746f14a92d5425d935e609571748e70 P 23bd656811574169affd015915dd21e5 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 23bd656811574169affd015915dd21e5; no voters: 440d47534a314ca88247d80185c766f8, 7ba1b776d4734e909008ca7c997fde5f
I20260809 15:39:22.680802 21638 raft_consensus.cc:2726] T 1746f14a92d5425d935e609571748e70 P 23bd656811574169affd015915dd21e5 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260809 15:39:22.732071 21642 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:39:22.731617 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.89.254:41873 (local address 127.3.89.195:44199)
0809 15:39:22.731893 (+ 276us) negotiation.cc:107] Waiting for socket to connect
0809 15:39:22.731957 (+ 64us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.89.254:41873: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":198,"thread_start_us":93,"threads_started":1}
I20260809 15:39:23.401957 21643 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:39:23.401543 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.89.254:41873 (local address 127.3.89.193:55251)
0809 15:39:23.401806 (+ 263us) negotiation.cc:107] Waiting for socket to connect
0809 15:39:23.401861 (+ 55us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.89.254:41873: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":179,"thread_start_us":80,"threads_started":1}
I20260809 15:39:23.550031 21644 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:39:23.549672 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.89.254:41873 (local address 127.3.89.194:37883)
0809 15:39:23.549895 (+ 223us) negotiation.cc:107] Waiting for socket to connect
0809 15:39:23.549949 (+ 54us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.89.254:41873: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":160,"thread_start_us":68,"threads_started":1}
I20260809 15:39:23.733243 21645 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:39:23.732828 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.89.254:41873 (local address 127.3.89.195:38595)
0809 15:39:23.733075 (+ 247us) negotiation.cc:107] Waiting for socket to connect
0809 15:39:23.733132 (+ 57us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.89.254:41873: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":178,"thread_start_us":77,"threads_started":1}
I20260809 15:39:24.403359 21646 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:39:24.402765 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.89.254:41873 (local address 127.3.89.193:55675)
0809 15:39:24.403192 (+ 427us) negotiation.cc:107] Waiting for socket to connect
0809 15:39:24.403248 (+ 56us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.89.254:41873: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":357,"thread_start_us":76,"threads_started":1}
I20260809 15:39:24.551214 21647 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:39:24.550777 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.89.254:41873 (local address 127.3.89.194:42967)
0809 15:39:24.551082 (+ 305us) negotiation.cc:107] Waiting for socket to connect
0809 15:39:24.551131 (+ 49us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.89.254:41873: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":241,"thread_start_us":90,"threads_started":1}
I20260809 15:39:24.734375 21648 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:39:24.733982 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.89.254:41873 (local address 127.3.89.195:40349)
0809 15:39:24.734235 (+ 253us) negotiation.cc:107] Waiting for socket to connect
0809 15:39:24.734289 (+ 54us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.89.254:41873: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":184,"thread_start_us":72,"threads_started":1}
I20260809 15:39:24.758318 21649 raft_consensus.cc:482] T 1746f14a92d5425d935e609571748e70 P 23bd656811574169affd015915dd21e5 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 15:39:24.758455 21649 raft_consensus.cc:504] T 1746f14a92d5425d935e609571748e70 P 23bd656811574169affd015915dd21e5 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7ba1b776d4734e909008ca7c997fde5f" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 32773 } } peers { permanent_uuid: "440d47534a314ca88247d80185c766f8" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 44003 } } peers { permanent_uuid: "23bd656811574169affd015915dd21e5" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 36955 } }
I20260809 15:39:24.759040 21649 leader_election.cc:288] T 1746f14a92d5425d935e609571748e70 P 23bd656811574169affd015915dd21e5 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 7ba1b776d4734e909008ca7c997fde5f (127.3.89.194:32773), 440d47534a314ca88247d80185c766f8 (127.3.89.195:44003)
W20260809 15:39:24.759872 21216 leader_election.cc:341] T 1746f14a92d5425d935e609571748e70 P 23bd656811574169affd015915dd21e5 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 7ba1b776d4734e909008ca7c997fde5f (127.3.89.194:32773): Not found: Tablet not found: 1746f14a92d5425d935e609571748e70
W20260809 15:39:24.760042 21215 leader_election.cc:341] T 1746f14a92d5425d935e609571748e70 P 23bd656811574169affd015915dd21e5 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 440d47534a314ca88247d80185c766f8 (127.3.89.195:44003): Not found: Tablet not found: 1746f14a92d5425d935e609571748e70
I20260809 15:39:24.760151 21215 leader_election.cc:302] T 1746f14a92d5425d935e609571748e70 P 23bd656811574169affd015915dd21e5 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 23bd656811574169affd015915dd21e5; no voters: 440d47534a314ca88247d80185c766f8, 7ba1b776d4734e909008ca7c997fde5f
I20260809 15:39:24.760329 21649 raft_consensus.cc:2726] T 1746f14a92d5425d935e609571748e70 P 23bd656811574169affd015915dd21e5 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260809 15:39:25.404485 21650 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:39:25.404049 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.89.254:41873 (local address 127.3.89.193:35591)
0809 15:39:25.404340 (+ 291us) negotiation.cc:107] Waiting for socket to connect
0809 15:39:25.404391 (+ 51us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.89.254:41873: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":215,"thread_start_us":92,"threads_started":1}
I20260809 15:39:25.552277 21651 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:39:25.551889 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.89.254:41873 (local address 127.3.89.194:58343)
0809 15:39:25.552139 (+ 250us) negotiation.cc:107] Waiting for socket to connect
0809 15:39:25.552195 (+ 56us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.89.254:41873: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":179,"thread_start_us":73,"threads_started":1}
I20260809 15:39:25.735556 21652 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:39:25.735153 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.89.254:41873 (local address 127.3.89.195:55907)
0809 15:39:25.735425 (+ 272us) negotiation.cc:107] Waiting for socket to connect
0809 15:39:25.735475 (+ 50us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.89.254:41873: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":197,"thread_start_us":108,"threads_started":1}
I20260809 15:39:26.405805 21653 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:39:26.405387 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.89.254:41873 (local address 127.3.89.193:50065)
0809 15:39:26.405660 (+ 273us) negotiation.cc:107] Waiting for socket to connect
0809 15:39:26.405711 (+ 51us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.89.254:41873: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":208,"thread_start_us":79,"threads_started":1}
I20260809 15:39:26.553393 21654 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:39:26.553015 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.89.254:41873 (local address 127.3.89.194:60191)
0809 15:39:26.553250 (+ 235us) negotiation.cc:107] Waiting for socket to connect
0809 15:39:26.553302 (+ 52us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.89.254:41873: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":168,"thread_start_us":72,"threads_started":1}
I20260809 15:39:26.736680 21655 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:39:26.736294 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.89.254:41873 (local address 127.3.89.195:46003)
0809 15:39:26.736538 (+ 244us) negotiation.cc:107] Waiting for socket to connect
0809 15:39:26.736590 (+ 52us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.89.254:41873: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":179,"thread_start_us":76,"threads_started":1}
I20260809 15:39:27.407038 21656 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:39:27.406526 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.89.254:41873 (local address 127.3.89.193:60595)
0809 15:39:27.406816 (+ 290us) negotiation.cc:107] Waiting for socket to connect
0809 15:39:27.406874 (+ 58us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.89.254:41873: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":210,"thread_start_us":78,"threads_started":1}
I20260809 15:39:27.540012 21657 raft_consensus.cc:482] T 1746f14a92d5425d935e609571748e70 P 23bd656811574169affd015915dd21e5 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 15:39:27.540143 21657 raft_consensus.cc:504] T 1746f14a92d5425d935e609571748e70 P 23bd656811574169affd015915dd21e5 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7ba1b776d4734e909008ca7c997fde5f" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 32773 } } peers { permanent_uuid: "440d47534a314ca88247d80185c766f8" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 44003 } } peers { permanent_uuid: "23bd656811574169affd015915dd21e5" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 36955 } }
I20260809 15:39:27.540635 21657 leader_election.cc:288] T 1746f14a92d5425d935e609571748e70 P 23bd656811574169affd015915dd21e5 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 7ba1b776d4734e909008ca7c997fde5f (127.3.89.194:32773), 440d47534a314ca88247d80185c766f8 (127.3.89.195:44003)
W20260809 15:39:27.541417 21216 leader_election.cc:341] T 1746f14a92d5425d935e609571748e70 P 23bd656811574169affd015915dd21e5 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 7ba1b776d4734e909008ca7c997fde5f (127.3.89.194:32773): Not found: Tablet not found: 1746f14a92d5425d935e609571748e70
W20260809 15:39:27.541555 21215 leader_election.cc:341] T 1746f14a92d5425d935e609571748e70 P 23bd656811574169affd015915dd21e5 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 440d47534a314ca88247d80185c766f8 (127.3.89.195:44003): Not found: Tablet not found: 1746f14a92d5425d935e609571748e70
I20260809 15:39:27.541661 21215 leader_election.cc:302] T 1746f14a92d5425d935e609571748e70 P 23bd656811574169affd015915dd21e5 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 23bd656811574169affd015915dd21e5; no voters: 440d47534a314ca88247d80185c766f8, 7ba1b776d4734e909008ca7c997fde5f
I20260809 15:39:27.541843 21657 raft_consensus.cc:2726] T 1746f14a92d5425d935e609571748e70 P 23bd656811574169affd015915dd21e5 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260809 15:39:27.554464 21658 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:39:27.554050 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.89.254:41873 (local address 127.3.89.194:42475)
0809 15:39:27.554327 (+ 277us) negotiation.cc:107] Waiting for socket to connect
0809 15:39:27.554379 (+ 52us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.89.254:41873: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":189,"thread_start_us":119,"threads_started":1}
I20260809 15:39:27.737794 21659 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:39:27.737389 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.89.254:41873 (local address 127.3.89.195:35239)
0809 15:39:27.737653 (+ 264us) negotiation.cc:107] Waiting for socket to connect
0809 15:39:27.737708 (+ 55us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.89.254:41873: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":195,"thread_start_us":88,"threads_started":1}
I20260809 15:39:28.408262 21660 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:39:28.407813 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.89.254:41873 (local address 127.3.89.193:48689)
0809 15:39:28.408106 (+ 293us) negotiation.cc:107] Waiting for socket to connect
0809 15:39:28.408165 (+ 59us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.89.254:41873: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":174,"thread_start_us":73,"threads_started":1}
I20260809 15:39:28.555586 21661 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:39:28.555150 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.89.254:41873 (local address 127.3.89.194:51783)
0809 15:39:28.555448 (+ 298us) negotiation.cc:107] Waiting for socket to connect
0809 15:39:28.555503 (+ 55us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.89.254:41873: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":230,"thread_start_us":85,"threads_started":1}
I20260809 15:39:28.739147 21662 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:39:28.738606 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.89.254:41873 (local address 127.3.89.195:47565)
0809 15:39:28.738927 (+ 321us) negotiation.cc:107] Waiting for socket to connect
0809 15:39:28.739047 (+ 120us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.89.254:41873: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":250,"thread_start_us":86,"threads_started":1}
I20260809 15:39:29.305256 21663 raft_consensus.cc:482] T 1746f14a92d5425d935e609571748e70 P 23bd656811574169affd015915dd21e5 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 15:39:29.305377 21663 raft_consensus.cc:504] T 1746f14a92d5425d935e609571748e70 P 23bd656811574169affd015915dd21e5 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7ba1b776d4734e909008ca7c997fde5f" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 32773 } } peers { permanent_uuid: "440d47534a314ca88247d80185c766f8" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 44003 } } peers { permanent_uuid: "23bd656811574169affd015915dd21e5" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 36955 } }
I20260809 15:39:29.305831 21663 leader_election.cc:288] T 1746f14a92d5425d935e609571748e70 P 23bd656811574169affd015915dd21e5 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 7ba1b776d4734e909008ca7c997fde5f (127.3.89.194:32773), 440d47534a314ca88247d80185c766f8 (127.3.89.195:44003)
W20260809 15:39:29.306613 21215 leader_election.cc:341] T 1746f14a92d5425d935e609571748e70 P 23bd656811574169affd015915dd21e5 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 440d47534a314ca88247d80185c766f8 (127.3.89.195:44003): Not found: Tablet not found: 1746f14a92d5425d935e609571748e70
W20260809 15:39:29.306725 21216 leader_election.cc:341] T 1746f14a92d5425d935e609571748e70 P 23bd656811574169affd015915dd21e5 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 7ba1b776d4734e909008ca7c997fde5f (127.3.89.194:32773): Not found: Tablet not found: 1746f14a92d5425d935e609571748e70
I20260809 15:39:29.306844 21216 leader_election.cc:302] T 1746f14a92d5425d935e609571748e70 P 23bd656811574169affd015915dd21e5 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 23bd656811574169affd015915dd21e5; no voters: 440d47534a314ca88247d80185c766f8, 7ba1b776d4734e909008ca7c997fde5f
I20260809 15:39:29.307053 21663 raft_consensus.cc:2726] T 1746f14a92d5425d935e609571748e70 P 23bd656811574169affd015915dd21e5 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260809 15:39:29.409411 21664 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:39:29.408947 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.89.254:41873 (local address 127.3.89.193:50915)
0809 15:39:29.409258 (+ 311us) negotiation.cc:107] Waiting for socket to connect
0809 15:39:29.409314 (+ 56us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.89.254:41873: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":230,"thread_start_us":114,"threads_started":1}
I20260809 15:39:29.556681 21665 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:39:29.556299 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.89.254:41873 (local address 127.3.89.194:48819)
0809 15:39:29.556540 (+ 241us) negotiation.cc:107] Waiting for socket to connect
0809 15:39:29.556594 (+ 54us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.89.254:41873: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":171,"thread_start_us":72,"threads_started":1}
I20260809 15:39:29.740291 21666 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:39:29.739870 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.89.254:41873 (local address 127.3.89.195:37901)
0809 15:39:29.740126 (+ 256us) negotiation.cc:107] Waiting for socket to connect
0809 15:39:29.740188 (+ 62us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.89.254:41873: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":180,"thread_start_us":70,"threads_started":1}
I20260809 15:39:30.410542 21667 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:39:30.410134 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.89.254:41873 (local address 127.3.89.193:43261)
0809 15:39:30.410381 (+ 247us) negotiation.cc:107] Waiting for socket to connect
0809 15:39:30.410435 (+ 54us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.89.254:41873: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":176,"thread_start_us":73,"threads_started":1}
I20260809 15:39:30.557852 21668 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:39:30.557418 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.89.254:41873 (local address 127.3.89.194:50025)
0809 15:39:30.557682 (+ 264us) negotiation.cc:107] Waiting for socket to connect
0809 15:39:30.557737 (+ 55us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.89.254:41873: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":192,"thread_start_us":92,"threads_started":1}
I20260809 15:39:30.741513 21669 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:39:30.741092 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.89.254:41873 (local address 127.3.89.195:43677)
0809 15:39:30.741367 (+ 275us) negotiation.cc:107] Waiting for socket to connect
0809 15:39:30.741423 (+ 56us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.89.254:41873: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":205,"thread_start_us":81,"threads_started":1}
I20260809 15:39:31.411852 21670 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:39:31.411409 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.89.254:41873 (local address 127.3.89.193:33795)
0809 15:39:31.411673 (+ 264us) negotiation.cc:107] Waiting for socket to connect
0809 15:39:31.411728 (+ 55us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.89.254:41873: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":194,"thread_start_us":94,"threads_started":1}
I20260809 15:39:31.559060 21671 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:39:31.558570 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.89.254:41873 (local address 127.3.89.194:43539)
0809 15:39:31.558860 (+ 290us) negotiation.cc:107] Waiting for socket to connect
0809 15:39:31.558916 (+ 56us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.89.254:41873: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":218,"thread_start_us":86,"threads_started":1}
I20260809 15:39:31.742782 21672 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:39:31.742365 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.89.254:41873 (local address 127.3.89.195:47161)
0809 15:39:31.742626 (+ 261us) negotiation.cc:107] Waiting for socket to connect
0809 15:39:31.742685 (+ 59us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.89.254:41873: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":192,"thread_start_us":93,"threads_started":1}
I20260809 15:39:32.413019 21673 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:39:32.412588 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.89.254:41873 (local address 127.3.89.193:34493)
0809 15:39:32.412856 (+ 268us) negotiation.cc:107] Waiting for socket to connect
0809 15:39:32.412912 (+ 56us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.89.254:41873: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":199,"thread_start_us":90,"threads_started":1}
I20260809 15:39:32.560212 21674 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:39:32.559785 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.89.254:41873 (local address 127.3.89.194:37071)
0809 15:39:32.560059 (+ 274us) negotiation.cc:107] Waiting for socket to connect
0809 15:39:32.560118 (+ 59us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.89.254:41873: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":203,"thread_start_us":79,"threads_started":1}
I20260809 15:39:32.743978 21675 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:39:32.743573 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.89.254:41873 (local address 127.3.89.195:34481)
0809 15:39:32.743833 (+ 260us) negotiation.cc:107] Waiting for socket to connect
0809 15:39:32.743886 (+ 53us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.89.254:41873: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":189,"thread_start_us":72,"threads_started":1}
I20260809 15:39:32.881906 21676 raft_consensus.cc:482] T 1746f14a92d5425d935e609571748e70 P 23bd656811574169affd015915dd21e5 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 15:39:32.882037 21676 raft_consensus.cc:504] T 1746f14a92d5425d935e609571748e70 P 23bd656811574169affd015915dd21e5 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7ba1b776d4734e909008ca7c997fde5f" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 32773 } } peers { permanent_uuid: "440d47534a314ca88247d80185c766f8" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 44003 } } peers { permanent_uuid: "23bd656811574169affd015915dd21e5" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 36955 } }
I20260809 15:39:32.882586 21676 leader_election.cc:288] T 1746f14a92d5425d935e609571748e70 P 23bd656811574169affd015915dd21e5 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 7ba1b776d4734e909008ca7c997fde5f (127.3.89.194:32773), 440d47534a314ca88247d80185c766f8 (127.3.89.195:44003)
W20260809 15:39:32.883423 21215 leader_election.cc:341] T 1746f14a92d5425d935e609571748e70 P 23bd656811574169affd015915dd21e5 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 440d47534a314ca88247d80185c766f8 (127.3.89.195:44003): Not found: Tablet not found: 1746f14a92d5425d935e609571748e70
W20260809 15:39:32.883515 21216 leader_election.cc:341] T 1746f14a92d5425d935e609571748e70 P 23bd656811574169affd015915dd21e5 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 7ba1b776d4734e909008ca7c997fde5f (127.3.89.194:32773): Not found: Tablet not found: 1746f14a92d5425d935e609571748e70
I20260809 15:39:32.883627 21216 leader_election.cc:302] T 1746f14a92d5425d935e609571748e70 P 23bd656811574169affd015915dd21e5 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 23bd656811574169affd015915dd21e5; no voters: 440d47534a314ca88247d80185c766f8, 7ba1b776d4734e909008ca7c997fde5f
I20260809 15:39:32.883780 21676 raft_consensus.cc:2726] T 1746f14a92d5425d935e609571748e70 P 23bd656811574169affd015915dd21e5 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260809 15:39:33.414198 21677 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:39:33.413783 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.89.254:41873 (local address 127.3.89.193:54433)
0809 15:39:33.414047 (+ 264us) negotiation.cc:107] Waiting for socket to connect
0809 15:39:33.414102 (+ 55us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.89.254:41873: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":193,"thread_start_us":77,"threads_started":1}
I20260809 15:39:33.561235 21678 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:39:33.560877 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.89.254:41873 (local address 127.3.89.194:40247)
0809 15:39:33.561100 (+ 223us) negotiation.cc:107] Waiting for socket to connect
0809 15:39:33.561149 (+ 49us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.89.254:41873: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":160,"thread_start_us":72,"threads_started":1}
I20260809 15:39:33.745244 21679 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:39:33.744866 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.89.254:41873 (local address 127.3.89.195:56289)
0809 15:39:33.745100 (+ 234us) negotiation.cc:107] Waiting for socket to connect
0809 15:39:33.745151 (+ 51us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.89.254:41873: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":168,"thread_start_us":77,"threads_started":1}
I20260809 15:39:34.415460 21680 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:39:34.415046 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.89.254:41873 (local address 127.3.89.193:38015)
0809 15:39:34.415304 (+ 258us) negotiation.cc:107] Waiting for socket to connect
0809 15:39:34.415362 (+ 58us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.89.254:41873: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":186,"thread_start_us":72,"threads_started":1}
I20260809 15:39:34.562364 21681 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:39:34.561973 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.89.254:41873 (local address 127.3.89.194:55593)
0809 15:39:34.562219 (+ 246us) negotiation.cc:107] Waiting for socket to connect
0809 15:39:34.562273 (+ 54us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.89.254:41873: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":177,"thread_start_us":75,"threads_started":1}
I20260809 15:39:34.746636 21682 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:39:34.746086 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.89.254:41873 (local address 127.3.89.195:39617)
0809 15:39:34.746427 (+ 341us) negotiation.cc:107] Waiting for socket to connect
0809 15:39:34.746513 (+ 86us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.89.254:41873: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":255,"thread_start_us":94,"threads_started":1}
I20260809 15:39:35.416710 21683 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:39:35.416243 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.89.254:41873 (local address 127.3.89.193:58547)
0809 15:39:35.416511 (+ 268us) negotiation.cc:107] Waiting for socket to connect
0809 15:39:35.416585 (+ 74us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.89.254:41873: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":177,"thread_start_us":80,"threads_started":1}
I20260809 15:39:35.563575 21684 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:39:35.563151 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.89.254:41873 (local address 127.3.89.194:39169)
0809 15:39:35.563413 (+ 262us) negotiation.cc:107] Waiting for socket to connect
0809 15:39:35.563477 (+ 64us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.89.254:41873: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":190,"thread_start_us":76,"threads_started":1}
I20260809 15:39:35.747807 21685 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:39:35.747376 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.89.254:41873 (local address 127.3.89.195:50755)
0809 15:39:35.747653 (+ 277us) negotiation.cc:107] Waiting for socket to connect
0809 15:39:35.747707 (+ 54us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.89.254:41873: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":207,"thread_start_us":107,"threads_started":1}
I20260809 15:39:36.418125 21686 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:39:36.417690 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.89.254:41873 (local address 127.3.89.193:39373)
0809 15:39:36.417972 (+ 282us) negotiation.cc:107] Waiting for socket to connect
0809 15:39:36.418024 (+ 52us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.89.254:41873: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":205,"thread_start_us":75,"threads_started":1}
I20260809 15:39:36.564815 21688 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:39:36.564441 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.89.254:41873 (local address 127.3.89.194:59009)
0809 15:39:36.564668 (+ 227us) negotiation.cc:107] Waiting for socket to connect
0809 15:39:36.564720 (+ 52us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.89.254:41873: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":161,"thread_start_us":72,"threads_started":1}
I20260809 15:39:36.730695 3431 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskm314YR/build/debug/bin/kudu with pid 21162
I20260809 15:39:36.738909 3431 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskm314YR/build/debug/bin/kudu with pid 21344
I20260809 15:39:36.746033 3431 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskm314YR/build/debug/bin/kudu with pid 21489
2026-08-09T15:39:36Z chronyd exiting
[ OK ] SecurityITest.TestEnableAndDisableEncryptedTSK (59424 ms)
[----------] 30 tests from SecurityITest (243898 ms total)
[----------] 6 tests from EncryptionPolicyTest
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/0
Loading random data
Initializing database '/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786289732823074-3431-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 15:39:36 dist-test-slave-2w3w krb5kdc[21691](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 15:39:36 dist-test-slave-2w3w krb5kdc[21691](info): set up 2 sockets
Aug 09 15:39:36 dist-test-slave-2w3w krb5kdc[21691](info): commencing operation
krb5kdc: starting...
W20260809 15:39:37.825625 3431 mini_kdc.cc:121] Time spent starting KDC: real 1.056s user 0.001s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786289732823074-3431-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786289732823074-3431-0/krb5kdc/test-user.keytab.
Aug 09 15:39:37 dist-test-slave-2w3w krb5kdc[21691](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786289977, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T15:39:37Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T15:39:37Z Disabled control of system clock
WARNING: no policy specified for kudu/127.3.89.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.3.89.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.3.89.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786289732823074-3431-0/krb5kdc/kudu_127.3.89.254.keytab.
Entry for principal kudu/127.3.89.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786289732823074-3431-0/krb5kdc/kudu_127.3.89.254.keytab.
I20260809 15:39:37.953671 3431 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskm314YR/build/debug/bin/kudu
/tmp/dist-test-taskm314YR/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786289732823074-3431-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786289732823074-3431-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786289732823074-3431-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786289732823074-3431-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.3.89.254:42961
--webserver_interface=127.3.89.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786289732823074-3431-0/krb5kdc/kudu_127.3.89.254.keytab
--principal=kudu/127.3.89.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.3.89.212:42875
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.3.89.254:42961
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786289732823074-3431-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786289732823074-3431-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786289732823074-3431-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 15:39:38.063362 21709 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 15:39:38.063640 21709 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 15:39:38.063722 21709 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 15:39:38.067703 21709 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 15:39:38.067814 21709 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 15:39:38.067857 21709 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 15:39:38.067914 21709 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 15:39:38.067953 21709 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 15:39:38.072995 21709 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.3.89.212:42875
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786289732823074-3431-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786289732823074-3431-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.3.89.254:42961
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786289732823074-3431-0/krb5kdc/kudu_127.3.89.254.keytab
--principal=kudu/127.3.89.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.3.89.254:42961
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786289732823074-3431-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.3.89.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.21709
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786289732823074-3431-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 09 Aug 2026 14:46:55 UTC on ccycloud.quasar-yswkmf.root.comops.site
build id 4576540
I20260809 15:39:38.073942 21709 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 15:39:38.074898 21709 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 15:39:38.083236 21718 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 15:39:38.083256 21715 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 15:39:38.083276 21716 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 15:39:38.083328 21709 server_base.cc:1034] running on GCE node
I20260809 15:39:38.084017 21709 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 15:39:38.085004 21709 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 15:39:38.086174 21709 hybrid_clock.cc:648] HybridClock initialized: now 1786289978086147 us; error 39 us; skew 500 ppm
Aug 09 15:39:38 dist-test-slave-2w3w krb5kdc[21691](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786289978, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.89.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 15:39:38.088910 21709 init.cc:377] Logged in from keytab as kudu/127.3.89.254@KRBTEST.COM (short username kudu)
I20260809 15:39:38.089862 21709 webserver.cc:460] Webserver started at http://127.3.89.254:37405/ using document root <none> and password file <none>
I20260809 15:39:38.090417 21709 fs_manager.cc:359] Metadata directory not provided
I20260809 15:39:38.090479 21709 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 15:39:38.090714 21709 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 15:39:38.092787 21709 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskm314YR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786289732823074-3431-0/minicluster-data/master-0/data/instance:
uuid: "1ee6dacea0274b3e9315e544bd186631"
format_stamp: "Formatted at 2026-08-09 15:39:38 on dist-test-slave-2w3w"
server_key: "dcb4796b40fc39b1bbd97854ff7de6a6"
server_key_iv: "d784c7e328b517e446e54ee156186b13"
server_key_version: "encryptionkey@0"
I20260809 15:39:38.093362 21709 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskm314YR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786289732823074-3431-0/minicluster-data/master-0/wal/instance:
uuid: "1ee6dacea0274b3e9315e544bd186631"
format_stamp: "Formatted at 2026-08-09 15:39:38 on dist-test-slave-2w3w"
server_key: "dcb4796b40fc39b1bbd97854ff7de6a6"
server_key_iv: "d784c7e328b517e446e54ee156186b13"
server_key_version: "encryptionkey@0"
I20260809 15:39:38.097034 21709 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.005s sys 0.000s
I20260809 15:39:38.099431 21726 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 15:39:38.100737 21709 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260809 15:39:38.100880 21709 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskm314YR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786289732823074-3431-0/minicluster-data/master-0/data,/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786289732823074-3431-0/minicluster-data/master-0/wal
uuid: "1ee6dacea0274b3e9315e544bd186631"
format_stamp: "Formatted at 2026-08-09 15:39:38 on dist-test-slave-2w3w"
server_key: "dcb4796b40fc39b1bbd97854ff7de6a6"
server_key_iv: "d784c7e328b517e446e54ee156186b13"
server_key_version: "encryptionkey@0"
I20260809 15:39:38.101004 21709 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskm314YR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786289732823074-3431-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskm314YR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786289732823074-3431-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskm314YR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786289732823074-3431-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 15:39:38.113893 21709 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 15:39:38.116827 21709 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 15:39:38.117024 21709 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 15:39:38.124353 21709 rpc_server.cc:307] RPC server started. Bound to: 127.3.89.254:42961
I20260809 15:39:38.124369 21781 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.3.89.254:42961 every 8 connection(s)
I20260809 15:39:38.125272 21709 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskm314YR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786289732823074-3431-0/minicluster-data/master-0/data/info.pb
I20260809 15:39:38.127943 21782 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 15:39:38.130549 3431 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskm314YR/build/debug/bin/kudu as pid 21709
I20260809 15:39:38.130649 3431 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskm314YR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786289732823074-3431-0/minicluster-data/master-0/wal/instance
I20260809 15:39:38.130993 3431 external_mini_cluster.cc:1428] Setting key f69e53416ad6139b91f3527ed557cc8c
I20260809 15:39:38.133991 21782 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1ee6dacea0274b3e9315e544bd186631: Bootstrap starting.
I20260809 15:39:38.136708 21782 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 1ee6dacea0274b3e9315e544bd186631: Neither blocks nor log segments found. Creating new log.
I20260809 15:39:38.137449 21782 log.cc:824] T 00000000000000000000000000000000 P 1ee6dacea0274b3e9315e544bd186631: Log is configured to *not* fsync() on all Append() calls
I20260809 15:39:38.139433 21782 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1ee6dacea0274b3e9315e544bd186631: No bootstrap required, opened a new log
Aug 09 15:39:38 dist-test-slave-2w3w krb5kdc[21691](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786289977, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.3.89.254@KRBTEST.COM
I20260809 15:39:38.143388 21782 raft_consensus.cc:348] T 00000000000000000000000000000000 P 1ee6dacea0274b3e9315e544bd186631 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1ee6dacea0274b3e9315e544bd186631" member_type: VOTER last_known_addr { host: "127.3.89.254" port: 42961 } }
I20260809 15:39:38.143676 21782 raft_consensus.cc:374] T 00000000000000000000000000000000 P 1ee6dacea0274b3e9315e544bd186631 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 15:39:38.143741 21782 raft_consensus.cc:729] T 00000000000000000000000000000000 P 1ee6dacea0274b3e9315e544bd186631 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1ee6dacea0274b3e9315e544bd186631, State: Initialized, Role: FOLLOWER
I20260809 15:39:38.144214 21782 consensus_queue.cc:260] T 00000000000000000000000000000000 P 1ee6dacea0274b3e9315e544bd186631 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1ee6dacea0274b3e9315e544bd186631" member_type: VOTER last_known_addr { host: "127.3.89.254" port: 42961 } }
I20260809 15:39:38.144394 21782 raft_consensus.cc:388] T 00000000000000000000000000000000 P 1ee6dacea0274b3e9315e544bd186631 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 15:39:38.144469 21782 raft_consensus.cc:482] T 00000000000000000000000000000000 P 1ee6dacea0274b3e9315e544bd186631 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 15:39:38.144565 21782 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 1ee6dacea0274b3e9315e544bd186631 [term 0 FOLLOWER]: Advancing to term 1
I20260809 15:39:38.145414 21782 raft_consensus.cc:504] T 00000000000000000000000000000000 P 1ee6dacea0274b3e9315e544bd186631 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1ee6dacea0274b3e9315e544bd186631" member_type: VOTER last_known_addr { host: "127.3.89.254" port: 42961 } }
I20260809 15:39:38.145788 21782 leader_election.cc:302] T 00000000000000000000000000000000 P 1ee6dacea0274b3e9315e544bd186631 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 1ee6dacea0274b3e9315e544bd186631; no voters:
I20260809 15:39:38.146101 21782 leader_election.cc:288] T 00000000000000000000000000000000 P 1ee6dacea0274b3e9315e544bd186631 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 15:39:38.146215 21787 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 1ee6dacea0274b3e9315e544bd186631 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 15:39:38.146432 21787 raft_consensus.cc:686] T 00000000000000000000000000000000 P 1ee6dacea0274b3e9315e544bd186631 [term 1 LEADER]: Becoming Leader. State: Replica: 1ee6dacea0274b3e9315e544bd186631, State: Running, Role: LEADER
I20260809 15:39:38.146632 21785 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:39:38.132385 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40820 (local address 127.3.89.254:42961)
0809 15:39:38.132799 (+ 414us) server_negotiation.cc:206] Beginning negotiation
0809 15:39:38.132818 (+ 19us) server_negotiation.cc:399] Waiting for connection header
0809 15:39:38.132862 (+ 44us) server_negotiation.cc:407] Connection header received
0809 15:39:38.133674 (+ 812us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 15:39:38.133697 (+ 23us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 15:39:38.134093 (+ 396us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 15:39:38.134448 (+ 355us) server_negotiation.cc:226] Negotiated authn=SASL
0809 15:39:38.135198 (+ 750us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:39:38.136692 (+ 1494us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:39:38.137599 (+ 907us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:39:38.138184 (+ 585us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:39:38.138356 (+ 172us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:39:38.140815 (+ 2459us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 15:39:38.140821 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 15:39:38.140836 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 15:39:38.140865 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0809 15:39:38.143488 (+ 2623us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 15:39:38.144012 (+ 524us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 15:39:38.144017 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 15:39:38.144023 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0809 15:39:38.144160 (+ 137us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 15:39:38.144478 (+ 318us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 15:39:38.144483 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 15:39:38.144488 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0809 15:39:38.144950 (+ 462us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 15:39:38.145169 (+ 219us) server_negotiation.cc:1035] Waiting for connection context
0809 15:39:38.145457 (+ 288us) server_negotiation.cc:299] Negotiation successful
0809 15:39:38.145737 (+ 280us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":261,"thread_start_us":103,"threads_started":1}
I20260809 15:39:38.146981 21787 consensus_queue.cc:237] T 00000000000000000000000000000000 P 1ee6dacea0274b3e9315e544bd186631 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1ee6dacea0274b3e9315e544bd186631" member_type: VOTER last_known_addr { host: "127.3.89.254" port: 42961 } }
I20260809 15:39:38.147357 21782 sys_catalog.cc:564] T 00000000000000000000000000000000 P 1ee6dacea0274b3e9315e544bd186631 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 15:39:38.148168 21787 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1ee6dacea0274b3e9315e544bd186631 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 1ee6dacea0274b3e9315e544bd186631. Latest consensus state: current_term: 1 leader_uuid: "1ee6dacea0274b3e9315e544bd186631" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1ee6dacea0274b3e9315e544bd186631" member_type: VOTER last_known_addr { host: "127.3.89.254" port: 42961 } } }
I20260809 15:39:38.148370 21787 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1ee6dacea0274b3e9315e544bd186631 [sys.catalog]: This master's current role is: LEADER
I20260809 15:39:38.148622 21788 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1ee6dacea0274b3e9315e544bd186631 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "1ee6dacea0274b3e9315e544bd186631" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1ee6dacea0274b3e9315e544bd186631" member_type: VOTER last_known_addr { host: "127.3.89.254" port: 42961 } } }
I20260809 15:39:38.148727 21788 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1ee6dacea0274b3e9315e544bd186631 [sys.catalog]: This master's current role is: LEADER
I20260809 15:39:38.148836 21796 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 15:39:38.151997 21796 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 15:39:38.157132 21796 catalog_manager.cc:1347] Generated new cluster ID: ed77e94c5cd04bcfb17e56a32a571873
I20260809 15:39:38.157194 21796 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 15:39:38.167187 21796 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 15:39:38.168087 21796 catalog_manager.cc:1504] Loading token signing keys...
I20260809 15:39:38.173516 21796 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 1ee6dacea0274b3e9315e544bd186631: Generated new TSK 0
I20260809 15:39:38.174124 21796 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.3.89.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.3.89.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.3.89.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786289732823074-3431-0/krb5kdc/kudu_127.3.89.193.keytab.
Entry for principal kudu/127.3.89.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786289732823074-3431-0/krb5kdc/kudu_127.3.89.193.keytab.
I20260809 15:39:38.203413 3431 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskm314YR/build/debug/bin/kudu
/tmp/dist-test-taskm314YR/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786289732823074-3431-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786289732823074-3431-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786289732823074-3431-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786289732823074-3431-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.3.89.193:0
--local_ip_for_outbound_sockets=127.3.89.193
--webserver_interface=127.3.89.193
--webserver_port=0
--tserver_master_addrs=127.3.89.254:42961
--keytab_file=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786289732823074-3431-0/krb5kdc/kudu_127.3.89.193.keytab
--principal=kudu/127.3.89.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.3.89.212:42875
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786289732823074-3431-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786289732823074-3431-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786289732823074-3431-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 15:39:38.313331 21808 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 15:39:38.313563 21808 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 15:39:38.313604 21808 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 15:39:38.317687 21808 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 15:39:38.317740 21808 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 15:39:38.317795 21808 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.3.89.193
I20260809 15:39:38.322815 21808 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.3.89.212:42875
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786289732823074-3431-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786289732823074-3431-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786289732823074-3431-0/krb5kdc/kudu_127.3.89.193.keytab
--principal=kudu/127.3.89.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.3.89.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786289732823074-3431-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.3.89.193
--webserver_port=0
--tserver_master_addrs=127.3.89.254:42961
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.21808
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.3.89.193
--log_dir=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786289732823074-3431-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 09 Aug 2026 14:46:55 UTC on ccycloud.quasar-yswkmf.root.comops.site
build id 4576540
I20260809 15:39:38.323810 21808 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 15:39:38.324724 21808 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 15:39:38.334009 21816 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 15:39:38.334165 21808 server_base.cc:1034] running on GCE node
W20260809 15:39:38.333983 21814 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 15:39:38.333982 21813 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 15:39:38.334692 21808 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 15:39:38.335402 21808 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 15:39:38.336561 21808 hybrid_clock.cc:648] HybridClock initialized: now 1786289978336534 us; error 41 us; skew 500 ppm
Aug 09 15:39:38 dist-test-slave-2w3w krb5kdc[21691](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786289978, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.89.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 15:39:38.339239 21808 init.cc:377] Logged in from keytab as kudu/127.3.89.193@KRBTEST.COM (short username kudu)
I20260809 15:39:38.340063 21808 webserver.cc:460] Webserver started at http://127.3.89.193:42023/ using document root <none> and password file <none>
I20260809 15:39:38.340621 21808 fs_manager.cc:359] Metadata directory not provided
I20260809 15:39:38.340740 21808 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 15:39:38.341068 21808 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 15:39:38.343380 21808 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskm314YR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786289732823074-3431-0/minicluster-data/ts-0/data/instance:
uuid: "6f745da6a1a34eb7b394dec2ef2cd47c"
format_stamp: "Formatted at 2026-08-09 15:39:38 on dist-test-slave-2w3w"
server_key: "88602aa994f91e71fbc91d8203401818"
server_key_iv: "152af7b2e87db8cf764180abe13773b7"
server_key_version: "encryptionkey@0"
I20260809 15:39:38.343969 21808 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskm314YR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786289732823074-3431-0/minicluster-data/ts-0/wal/instance:
uuid: "6f745da6a1a34eb7b394dec2ef2cd47c"
format_stamp: "Formatted at 2026-08-09 15:39:38 on dist-test-slave-2w3w"
server_key: "88602aa994f91e71fbc91d8203401818"
server_key_iv: "152af7b2e87db8cf764180abe13773b7"
server_key_version: "encryptionkey@0"
I20260809 15:39:38.347536 21808 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.003s sys 0.000s
I20260809 15:39:38.350054 21824 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 15:39:38.351260 21808 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.000s sys 0.003s
I20260809 15:39:38.351352 21808 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskm314YR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786289732823074-3431-0/minicluster-data/ts-0/data,/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786289732823074-3431-0/minicluster-data/ts-0/wal
uuid: "6f745da6a1a34eb7b394dec2ef2cd47c"
format_stamp: "Formatted at 2026-08-09 15:39:38 on dist-test-slave-2w3w"
server_key: "88602aa994f91e71fbc91d8203401818"
server_key_iv: "152af7b2e87db8cf764180abe13773b7"
server_key_version: "encryptionkey@0"
I20260809 15:39:38.351431 21808 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskm314YR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786289732823074-3431-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskm314YR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786289732823074-3431-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskm314YR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786289732823074-3431-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 15:39:38.369536 21808 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 15:39:38.372272 21808 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 15:39:38.372421 21808 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 15:39:38.372918 21808 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 15:39:38.373733 21808 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 15:39:38.373787 21808 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 15:39:38.373826 21808 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 15:39:38.373842 21808 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 15:39:38.382767 21808 rpc_server.cc:307] RPC server started. Bound to: 127.3.89.193:39593
I20260809 15:39:38.382799 21947 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.3.89.193:39593 every 8 connection(s)
I20260809 15:39:38.383654 21808 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskm314YR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786289732823074-3431-0/minicluster-data/ts-0/data/info.pb
I20260809 15:39:38.389986 3431 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskm314YR/build/debug/bin/kudu as pid 21808
I20260809 15:39:38.390091 3431 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskm314YR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786289732823074-3431-0/minicluster-data/ts-0/wal/instance
I20260809 15:39:38.390424 3431 external_mini_cluster.cc:1428] Setting key a24a0083bed3345bd1e337a8296a3232
Aug 09 15:39:38 dist-test-slave-2w3w krb5kdc[21691](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786289978, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.89.193@KRBTEST.COM for kudu/127.3.89.254@KRBTEST.COM
I20260809 15:39:38.397505 21785 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:39:38.385405 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.89.193:51155 (local address 127.3.89.254:42961)
0809 15:39:38.385558 (+ 153us) server_negotiation.cc:206] Beginning negotiation
0809 15:39:38.385563 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 15:39:38.386436 (+ 873us) server_negotiation.cc:407] Connection header received
0809 15:39:38.387706 (+ 1270us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 15:39:38.387708 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 15:39:38.387755 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 15:39:38.387836 (+ 81us) server_negotiation.cc:226] Negotiated authn=SASL
0809 15:39:38.388920 (+ 1084us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:39:38.389451 (+ 531us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:39:38.390728 (+ 1277us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:39:38.391085 (+ 357us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:39:38.391203 (+ 118us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:39:38.393841 (+ 2638us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 15:39:38.393845 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 15:39:38.393848 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 15:39:38.393874 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0809 15:39:38.395421 (+ 1547us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 15:39:38.395947 (+ 526us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 15:39:38.395952 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 15:39:38.395954 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 15:39:38.396041 (+ 87us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 15:39:38.396395 (+ 354us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 15:39:38.396399 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 15:39:38.396400 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 15:39:38.396595 (+ 195us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 15:39:38.396711 (+ 116us) server_negotiation.cc:1035] Waiting for connection context
0809 15:39:38.397195 (+ 484us) server_negotiation.cc:299] Negotiation successful
0809 15:39:38.397324 (+ 129us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":56}
I20260809 15:39:38.398339 21950 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:39:38.385793 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.89.254:42961 (local address 127.3.89.193:51155)
0809 15:39:38.386271 (+ 478us) negotiation.cc:107] Waiting for socket to connect
0809 15:39:38.386324 (+ 53us) client_negotiation.cc:174] Beginning negotiation
0809 15:39:38.387521 (+ 1197us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 15:39:38.387998 (+ 477us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 15:39:38.388004 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 15:39:38.388336 (+ 332us) client_negotiation.cc:189] Negotiated authn=SASL
0809 15:39:38.388785 (+ 449us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:39:38.388797 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:39:38.389559 (+ 762us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:39:38.389561 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:39:38.390602 (+ 1041us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:39:38.390612 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:39:38.391241 (+ 629us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:39:38.391245 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:39:38.391417 (+ 172us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:39:38.392051 (+ 634us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 15:39:38.392066 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0809 15:39:38.393656 (+ 1590us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 15:39:38.395555 (+ 1899us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 15:39:38.395561 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 15:39:38.395574 (+ 13us) client_negotiation.cc:760] calling sasl_client_step()
0809 15:39:38.395836 (+ 262us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 15:39:38.396140 (+ 304us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 15:39:38.396148 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 15:39:38.396150 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 15:39:38.396293 (+ 143us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 15:39:38.396711 (+ 418us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 15:39:38.396720 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 15:39:38.396953 (+ 233us) client_negotiation.cc:769] Sending connection context
0809 15:39:38.397151 (+ 198us) client_negotiation.cc:240] Negotiation successful
0809 15:39:38.397459 (+ 308us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":282,"thread_start_us":146,"threads_started":1}
I20260809 15:39:38.399639 21948 heartbeater.cc:347] Connected to a master server at 127.3.89.254:42961
I20260809 15:39:38.399878 21948 heartbeater.cc:464] Registering TS with master...
I20260809 15:39:38.400403 21948 heartbeater.cc:511] Master 127.3.89.254:42961 requested a full tablet report, sending...
WARNING: no policy specified for kudu/127.3.89.194@KRBTEST.COM; defaulting to no policy
I20260809 15:39:38.401943 21744 ts_manager.cc:194] Registered new tserver with Master: 6f745da6a1a34eb7b394dec2ef2cd47c (127.3.89.193:39593)
I20260809 15:39:38.403182 21744 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.3.89.193@KRBTEST.COM'} at 127.3.89.193:51155
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.3.89.194@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.3.89.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786289732823074-3431-0/krb5kdc/kudu_127.3.89.194.keytab.
Entry for principal kudu/127.3.89.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786289732823074-3431-0/krb5kdc/kudu_127.3.89.194.keytab.
I20260809 15:39:38.420017 3431 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskm314YR/build/debug/bin/kudu
/tmp/dist-test-taskm314YR/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786289732823074-3431-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786289732823074-3431-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786289732823074-3431-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786289732823074-3431-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.3.89.194:0
--local_ip_for_outbound_sockets=127.3.89.194
--webserver_interface=127.3.89.194
--webserver_port=0
--tserver_master_addrs=127.3.89.254:42961
--keytab_file=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786289732823074-3431-0/krb5kdc/kudu_127.3.89.194.keytab
--principal=kudu/127.3.89.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.3.89.212:42875
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786289732823074-3431-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786289732823074-3431-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786289732823074-3431-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 15:39:38.530210 21953 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 15:39:38.530484 21953 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 15:39:38.530565 21953 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 15:39:38.534586 21953 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 15:39:38.534672 21953 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 15:39:38.534771 21953 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.3.89.194
I20260809 15:39:38.540069 21953 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.3.89.212:42875
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786289732823074-3431-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786289732823074-3431-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786289732823074-3431-0/krb5kdc/kudu_127.3.89.194.keytab
--principal=kudu/127.3.89.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.3.89.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786289732823074-3431-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.3.89.194
--webserver_port=0
--tserver_master_addrs=127.3.89.254:42961
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.21953
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.3.89.194
--log_dir=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786289732823074-3431-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 09 Aug 2026 14:46:55 UTC on ccycloud.quasar-yswkmf.root.comops.site
build id 4576540
I20260809 15:39:38.541090 21953 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 15:39:38.542073 21953 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 15:39:38.551481 21961 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 15:39:38.551504 21958 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 15:39:38.551503 21959 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 15:39:38.551856 21953 server_base.cc:1034] running on GCE node
I20260809 15:39:38.552361 21953 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 15:39:38.553082 21953 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 15:39:38.554275 21953 hybrid_clock.cc:648] HybridClock initialized: now 1786289978554257 us; error 32 us; skew 500 ppm
Aug 09 15:39:38 dist-test-slave-2w3w krb5kdc[21691](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786289978, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.89.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 15:39:38.557067 21953 init.cc:377] Logged in from keytab as kudu/127.3.89.194@KRBTEST.COM (short username kudu)
I20260809 15:39:38.557822 21953 webserver.cc:460] Webserver started at http://127.3.89.194:45199/ using document root <none> and password file <none>
I20260809 15:39:38.558306 21953 fs_manager.cc:359] Metadata directory not provided
I20260809 15:39:38.558362 21953 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 15:39:38.558514 21953 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 15:39:38.560499 21953 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskm314YR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786289732823074-3431-0/minicluster-data/ts-1/data/instance:
uuid: "fe24e053bd2d42038b87676819285063"
format_stamp: "Formatted at 2026-08-09 15:39:38 on dist-test-slave-2w3w"
server_key: "3bb018210f21ceeadd96ae5f39dfc0ea"
server_key_iv: "fbbb42ab04bd51d24d6a6d8fa42966b6"
server_key_version: "encryptionkey@0"
I20260809 15:39:38.560974 21953 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskm314YR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786289732823074-3431-0/minicluster-data/ts-1/wal/instance:
uuid: "fe24e053bd2d42038b87676819285063"
format_stamp: "Formatted at 2026-08-09 15:39:38 on dist-test-slave-2w3w"
server_key: "3bb018210f21ceeadd96ae5f39dfc0ea"
server_key_iv: "fbbb42ab04bd51d24d6a6d8fa42966b6"
server_key_version: "encryptionkey@0"
I20260809 15:39:38.564379 21953 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.003s sys 0.002s
I20260809 15:39:38.566612 21968 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 15:39:38.567799 21953 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260809 15:39:38.567931 21953 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskm314YR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786289732823074-3431-0/minicluster-data/ts-1/data,/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786289732823074-3431-0/minicluster-data/ts-1/wal
uuid: "fe24e053bd2d42038b87676819285063"
format_stamp: "Formatted at 2026-08-09 15:39:38 on dist-test-slave-2w3w"
server_key: "3bb018210f21ceeadd96ae5f39dfc0ea"
server_key_iv: "fbbb42ab04bd51d24d6a6d8fa42966b6"
server_key_version: "encryptionkey@0"
I20260809 15:39:38.568030 21953 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskm314YR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786289732823074-3431-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskm314YR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786289732823074-3431-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskm314YR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786289732823074-3431-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 15:39:38.589255 21953 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 15:39:38.592360 21953 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 15:39:38.592571 21953 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 15:39:38.593214 21953 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 15:39:38.594063 21953 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 15:39:38.594111 21953 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 15:39:38.594182 21953 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 15:39:38.594225 21953 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 15:39:38.603066 21953 rpc_server.cc:307] RPC server started. Bound to: 127.3.89.194:46783
I20260809 15:39:38.603093 22094 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.3.89.194:46783 every 8 connection(s)
I20260809 15:39:38.603921 21953 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskm314YR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786289732823074-3431-0/minicluster-data/ts-1/data/info.pb
I20260809 15:39:38.606930 3431 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskm314YR/build/debug/bin/kudu as pid 21953
I20260809 15:39:38.607089 3431 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskm314YR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786289732823074-3431-0/minicluster-data/ts-1/wal/instance
I20260809 15:39:38.607411 3431 external_mini_cluster.cc:1428] Setting key 119a320b250be4c0f7bc847513f5eac0
Aug 09 15:39:38 dist-test-slave-2w3w krb5kdc[21691](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786289978, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.89.194@KRBTEST.COM for kudu/127.3.89.254@KRBTEST.COM
I20260809 15:39:38.618670 21785 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:39:38.605630 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.89.194:60295 (local address 127.3.89.254:42961)
0809 15:39:38.605783 (+ 153us) server_negotiation.cc:206] Beginning negotiation
0809 15:39:38.605786 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0809 15:39:38.606625 (+ 839us) server_negotiation.cc:407] Connection header received
0809 15:39:38.608171 (+ 1546us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 15:39:38.608174 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 15:39:38.608234 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 15:39:38.608336 (+ 102us) server_negotiation.cc:226] Negotiated authn=SASL
0809 15:39:38.609587 (+ 1251us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:39:38.610110 (+ 523us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:39:38.611705 (+ 1595us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:39:38.611935 (+ 230us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:39:38.612021 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:39:38.614812 (+ 2791us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 15:39:38.614819 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 15:39:38.614825 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 15:39:38.614861 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0809 15:39:38.616564 (+ 1703us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 15:39:38.617042 (+ 478us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 15:39:38.617047 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 15:39:38.617049 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 15:39:38.617146 (+ 97us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 15:39:38.617506 (+ 360us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 15:39:38.617509 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 15:39:38.617511 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 15:39:38.617768 (+ 257us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 15:39:38.617905 (+ 137us) server_negotiation.cc:1035] Waiting for connection context
0809 15:39:38.618357 (+ 452us) server_negotiation.cc:299] Negotiation successful
0809 15:39:38.618514 (+ 157us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":62}
WARNING: no policy specified for kudu/127.3.89.195@KRBTEST.COM; defaulting to no policy
I20260809 15:39:38.619411 22098 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:39:38.606054 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.89.254:42961 (local address 127.3.89.194:60295)
0809 15:39:38.606475 (+ 421us) negotiation.cc:107] Waiting for socket to connect
0809 15:39:38.606517 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0809 15:39:38.607941 (+ 1424us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 15:39:38.608503 (+ 562us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 15:39:38.608509 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 15:39:38.608955 (+ 446us) client_negotiation.cc:189] Negotiated authn=SASL
0809 15:39:38.609435 (+ 480us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:39:38.609447 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:39:38.610239 (+ 792us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:39:38.610243 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:39:38.611572 (+ 1329us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:39:38.611587 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:39:38.612090 (+ 503us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:39:38.612097 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:39:38.612316 (+ 219us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:39:38.612901 (+ 585us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 15:39:38.612925 (+ 24us) client_negotiation.cc:656] calling sasl_client_start()
0809 15:39:38.614630 (+ 1705us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 15:39:38.616674 (+ 2044us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 15:39:38.616678 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 15:39:38.616687 (+ 9us) client_negotiation.cc:760] calling sasl_client_step()
0809 15:39:38.616937 (+ 250us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 15:39:38.617247 (+ 310us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 15:39:38.617249 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 15:39:38.617250 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0809 15:39:38.617399 (+ 149us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 15:39:38.617896 (+ 497us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 15:39:38.617900 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 15:39:38.618109 (+ 209us) client_negotiation.cc:769] Sending connection context
0809 15:39:38.618255 (+ 146us) client_negotiation.cc:240] Negotiation successful
0809 15:39:38.618511 (+ 256us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":247,"thread_start_us":103,"threads_started":1}
I20260809 15:39:38.620597 22095 heartbeater.cc:347] Connected to a master server at 127.3.89.254:42961
I20260809 15:39:38.620839 22095 heartbeater.cc:464] Registering TS with master...
I20260809 15:39:38.621336 22095 heartbeater.cc:511] Master 127.3.89.254:42961 requested a full tablet report, sending...
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.3.89.195@KRBTEST.COM" created.
I20260809 15:39:38.622400 21744 ts_manager.cc:194] Registered new tserver with Master: fe24e053bd2d42038b87676819285063 (127.3.89.194:46783)
I20260809 15:39:38.623080 21744 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.3.89.194@KRBTEST.COM'} at 127.3.89.194:60295
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.3.89.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786289732823074-3431-0/krb5kdc/kudu_127.3.89.195.keytab.
Entry for principal kudu/127.3.89.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786289732823074-3431-0/krb5kdc/kudu_127.3.89.195.keytab.
I20260809 15:39:38.638028 3431 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskm314YR/build/debug/bin/kudu
/tmp/dist-test-taskm314YR/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786289732823074-3431-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786289732823074-3431-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786289732823074-3431-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786289732823074-3431-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.3.89.195:0
--local_ip_for_outbound_sockets=127.3.89.195
--webserver_interface=127.3.89.195
--webserver_port=0
--tserver_master_addrs=127.3.89.254:42961
--keytab_file=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786289732823074-3431-0/krb5kdc/kudu_127.3.89.195.keytab
--principal=kudu/127.3.89.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.3.89.212:42875
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786289732823074-3431-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786289732823074-3431-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786289732823074-3431-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 15:39:38.747466 22101 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 15:39:38.747787 22101 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 15:39:38.747848 22101 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 15:39:38.751897 22101 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 15:39:38.751950 22101 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 15:39:38.752040 22101 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.3.89.195
I20260809 15:39:38.757318 22101 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.3.89.212:42875
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786289732823074-3431-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786289732823074-3431-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786289732823074-3431-0/krb5kdc/kudu_127.3.89.195.keytab
--principal=kudu/127.3.89.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.3.89.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786289732823074-3431-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.3.89.195
--webserver_port=0
--tserver_master_addrs=127.3.89.254:42961
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.22101
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.3.89.195
--log_dir=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786289732823074-3431-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 09 Aug 2026 14:46:55 UTC on ccycloud.quasar-yswkmf.root.comops.site
build id 4576540
I20260809 15:39:38.758359 22101 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 15:39:38.759399 22101 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 15:39:38.768772 22109 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 15:39:38.768805 22107 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 15:39:38.768774 22106 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 15:39:38.769006 22101 server_base.cc:1034] running on GCE node
I20260809 15:39:38.769518 22101 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 15:39:38.770117 22101 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 15:39:38.771293 22101 hybrid_clock.cc:648] HybridClock initialized: now 1786289978771286 us; error 29 us; skew 500 ppm
Aug 09 15:39:38 dist-test-slave-2w3w krb5kdc[21691](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786289978, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.89.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 15:39:38.777328 22101 init.cc:377] Logged in from keytab as kudu/127.3.89.195@KRBTEST.COM (short username kudu)
I20260809 15:39:38.778232 22101 webserver.cc:460] Webserver started at http://127.3.89.195:44777/ using document root <none> and password file <none>
I20260809 15:39:38.778784 22101 fs_manager.cc:359] Metadata directory not provided
I20260809 15:39:38.778894 22101 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 15:39:38.779141 22101 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 15:39:38.781075 22101 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskm314YR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786289732823074-3431-0/minicluster-data/ts-2/data/instance:
uuid: "3b99c4280a9c48f5a293ad7e44e9ab5e"
format_stamp: "Formatted at 2026-08-09 15:39:38 on dist-test-slave-2w3w"
server_key: "a77ee2f72e4c5302a5a28e3219037d99"
server_key_iv: "e4122ec49e801f5b3bf4a8c110c83e2d"
server_key_version: "encryptionkey@0"
I20260809 15:39:38.781646 22101 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskm314YR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786289732823074-3431-0/minicluster-data/ts-2/wal/instance:
uuid: "3b99c4280a9c48f5a293ad7e44e9ab5e"
format_stamp: "Formatted at 2026-08-09 15:39:38 on dist-test-slave-2w3w"
server_key: "a77ee2f72e4c5302a5a28e3219037d99"
server_key_iv: "e4122ec49e801f5b3bf4a8c110c83e2d"
server_key_version: "encryptionkey@0"
I20260809 15:39:38.785537 22101 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.006s sys 0.000s
I20260809 15:39:38.787928 22118 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 15:39:38.789064 22101 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260809 15:39:38.789206 22101 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskm314YR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786289732823074-3431-0/minicluster-data/ts-2/data,/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786289732823074-3431-0/minicluster-data/ts-2/wal
uuid: "3b99c4280a9c48f5a293ad7e44e9ab5e"
format_stamp: "Formatted at 2026-08-09 15:39:38 on dist-test-slave-2w3w"
server_key: "a77ee2f72e4c5302a5a28e3219037d99"
server_key_iv: "e4122ec49e801f5b3bf4a8c110c83e2d"
server_key_version: "encryptionkey@0"
I20260809 15:39:38.789315 22101 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskm314YR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786289732823074-3431-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskm314YR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786289732823074-3431-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskm314YR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786289732823074-3431-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 15:39:38.802141 22101 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 15:39:38.805436 22101 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 15:39:38.805629 22101 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 15:39:38.806207 22101 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 15:39:38.807044 22101 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 15:39:38.807121 22101 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 15:39:38.807191 22101 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 15:39:38.807235 22101 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 15:39:38.816387 22101 rpc_server.cc:307] RPC server started. Bound to: 127.3.89.195:40017
I20260809 15:39:38.816414 22242 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.3.89.195:40017 every 8 connection(s)
I20260809 15:39:38.817250 22101 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskm314YR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786289732823074-3431-0/minicluster-data/ts-2/data/info.pb
I20260809 15:39:38.824571 3431 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskm314YR/build/debug/bin/kudu as pid 22101
I20260809 15:39:38.824684 3431 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskm314YR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786289732823074-3431-0/minicluster-data/ts-2/wal/instance
I20260809 15:39:38.825023 3431 external_mini_cluster.cc:1428] Setting key 8d54c8dd046679288f88a418332957b3
Aug 09 15:39:38 dist-test-slave-2w3w krb5kdc[21691](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786289978, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.89.195@KRBTEST.COM for kudu/127.3.89.254@KRBTEST.COM
I20260809 15:39:38.830222 21785 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:39:38.819018 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.89.195:36331 (local address 127.3.89.254:42961)
0809 15:39:38.819184 (+ 166us) server_negotiation.cc:206] Beginning negotiation
0809 15:39:38.819189 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 15:39:38.820039 (+ 850us) server_negotiation.cc:407] Connection header received
0809 15:39:38.821225 (+ 1186us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 15:39:38.821228 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 15:39:38.821275 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 15:39:38.821350 (+ 75us) server_negotiation.cc:226] Negotiated authn=SASL
0809 15:39:38.822435 (+ 1085us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:39:38.822947 (+ 512us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:39:38.824027 (+ 1080us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:39:38.824240 (+ 213us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:39:38.824335 (+ 95us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:39:38.826941 (+ 2606us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 15:39:38.826944 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 15:39:38.826946 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 15:39:38.827008 (+ 62us) server_negotiation.cc:922] calling sasl_server_start()
0809 15:39:38.828211 (+ 1203us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 15:39:38.828729 (+ 518us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 15:39:38.828737 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 15:39:38.828739 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 15:39:38.828820 (+ 81us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 15:39:38.829175 (+ 355us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 15:39:38.829179 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 15:39:38.829181 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 15:39:38.829369 (+ 188us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 15:39:38.829484 (+ 115us) server_negotiation.cc:1035] Waiting for connection context
0809 15:39:38.829922 (+ 438us) server_negotiation.cc:299] Negotiation successful
0809 15:39:38.830037 (+ 115us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":59}
I20260809 15:39:38.830929 22246 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:39:38.819389 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.89.254:42961 (local address 127.3.89.195:36331)
0809 15:39:38.819879 (+ 490us) negotiation.cc:107] Waiting for socket to connect
0809 15:39:38.819919 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0809 15:39:38.821029 (+ 1110us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 15:39:38.821499 (+ 470us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 15:39:38.821505 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 15:39:38.821865 (+ 360us) client_negotiation.cc:189] Negotiated authn=SASL
0809 15:39:38.822310 (+ 445us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:39:38.822321 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:39:38.823047 (+ 726us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:39:38.823049 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:39:38.823915 (+ 866us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:39:38.823924 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:39:38.824410 (+ 486us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:39:38.824416 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:39:38.824615 (+ 199us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:39:38.825261 (+ 646us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 15:39:38.825279 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0809 15:39:38.826780 (+ 1501us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 15:39:38.828350 (+ 1570us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 15:39:38.828356 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 15:39:38.828368 (+ 12us) client_negotiation.cc:760] calling sasl_client_step()
0809 15:39:38.828615 (+ 247us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 15:39:38.828919 (+ 304us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 15:39:38.828922 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 15:39:38.828924 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 15:39:38.829077 (+ 153us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 15:39:38.829482 (+ 405us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 15:39:38.829488 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 15:39:38.829703 (+ 215us) client_negotiation.cc:769] Sending connection context
0809 15:39:38.829878 (+ 175us) client_negotiation.cc:240] Negotiation successful
0809 15:39:38.830186 (+ 308us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":258,"thread_start_us":105,"threads_started":1}
I20260809 15:39:38.832065 22244 heartbeater.cc:347] Connected to a master server at 127.3.89.254:42961
I20260809 15:39:38.832327 22244 heartbeater.cc:464] Registering TS with master...
I20260809 15:39:38.832845 22244 heartbeater.cc:511] Master 127.3.89.254:42961 requested a full tablet report, sending...
I20260809 15:39:38.833881 21744 ts_manager.cc:194] Registered new tserver with Master: 3b99c4280a9c48f5a293ad7e44e9ab5e (127.3.89.195:40017)
I20260809 15:39:38.834434 21744 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.3.89.195@KRBTEST.COM'} at 127.3.89.195:36331
I20260809 15:39:38.838711 3431 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260809 15:39:38.848577 21785 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:39:38.840760 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49724 (local address 127.3.89.254:42961)
0809 15:39:38.840896 (+ 136us) server_negotiation.cc:206] Beginning negotiation
0809 15:39:38.840903 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 15:39:38.841152 (+ 249us) server_negotiation.cc:407] Connection header received
0809 15:39:38.841236 (+ 84us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 15:39:38.841238 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 15:39:38.841287 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 15:39:38.841352 (+ 65us) server_negotiation.cc:226] Negotiated authn=SASL
0809 15:39:38.841992 (+ 640us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:39:38.842487 (+ 495us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:39:38.843564 (+ 1077us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:39:38.843766 (+ 202us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:39:38.843854 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:39:38.845223 (+ 1369us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 15:39:38.845226 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 15:39:38.845228 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 15:39:38.845248 (+ 20us) server_negotiation.cc:922] calling sasl_server_start()
0809 15:39:38.846512 (+ 1264us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 15:39:38.847065 (+ 553us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 15:39:38.847068 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 15:39:38.847069 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 15:39:38.847140 (+ 71us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 15:39:38.847450 (+ 310us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 15:39:38.847453 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 15:39:38.847454 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 15:39:38.847658 (+ 204us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 15:39:38.847771 (+ 113us) server_negotiation.cc:1035] Waiting for connection context
0809 15:39:38.848238 (+ 467us) server_negotiation.cc:299] Negotiation successful
0809 15:39:38.848375 (+ 137us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":48}
I20260809 15:39:38.851470 21744 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:49724:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260809 15:39:38.854403 21744 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260809 15:39:38.868695 22255 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:39:38.862657 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.89.194:46783 (local address 127.0.0.1:41854)
0809 15:39:38.863310 (+ 653us) negotiation.cc:107] Waiting for socket to connect
0809 15:39:38.863327 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0809 15:39:38.863470 (+ 143us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 15:39:38.864232 (+ 762us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 15:39:38.864235 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 15:39:38.864253 (+ 18us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 15:39:38.864402 (+ 149us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:39:38.864410 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:39:38.865777 (+ 1367us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:39:38.865789 (+ 12us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:39:38.867269 (+ 1480us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:39:38.867280 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:39:38.868175 (+ 895us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:39:38.868178 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:39:38.868285 (+ 107us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:39:38.868319 (+ 34us) client_negotiation.cc:769] Sending connection context
0809 15:39:38.868429 (+ 110us) client_negotiation.cc:240] Negotiation successful
0809 15:39:38.868553 (+ 124us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":572,"mutex_wait_us":53,"spinlock_wait_cycles":6400,"thread_start_us":100,"threads_started":1}
I20260809 15:39:38.868703 22260 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:39:38.863416 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41854 (local address 127.3.89.194:46783)
0809 15:39:38.863768 (+ 352us) server_negotiation.cc:206] Beginning negotiation
0809 15:39:38.863777 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0809 15:39:38.863797 (+ 20us) server_negotiation.cc:407] Connection header received
0809 15:39:38.863883 (+ 86us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 15:39:38.863889 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 15:39:38.864071 (+ 182us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 15:39:38.864703 (+ 632us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 15:39:38.864868 (+ 165us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:39:38.865649 (+ 781us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:39:38.867396 (+ 1747us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:39:38.868043 (+ 647us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:39:38.868174 (+ 131us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:39:38.868325 (+ 151us) server_negotiation.cc:1035] Waiting for connection context
0809 15:39:38.868455 (+ 130us) server_negotiation.cc:299] Negotiation successful
0809 15:39:38.868549 (+ 94us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":237,"thread_start_us":130,"threads_started":1}
I20260809 15:39:38.869498 22261 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:39:38.863469 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33496 (local address 127.3.89.193:39593)
0809 15:39:38.864349 (+ 880us) server_negotiation.cc:206] Beginning negotiation
0809 15:39:38.864366 (+ 17us) server_negotiation.cc:399] Waiting for connection header
0809 15:39:38.864387 (+ 21us) server_negotiation.cc:407] Connection header received
0809 15:39:38.864458 (+ 71us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 15:39:38.864463 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 15:39:38.864675 (+ 212us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 15:39:38.864831 (+ 156us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 15:39:38.865056 (+ 225us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:39:38.866098 (+ 1042us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:39:38.867580 (+ 1482us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:39:38.868364 (+ 784us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:39:38.868468 (+ 104us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:39:38.868597 (+ 129us) server_negotiation.cc:1035] Waiting for connection context
0809 15:39:38.869240 (+ 643us) server_negotiation.cc:299] Negotiation successful
0809 15:39:38.869365 (+ 125us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":780,"thread_start_us":133,"threads_started":1}
I20260809 15:39:38.871291 22259 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:39:38.862651 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40294 (local address 127.3.89.195:40017)
0809 15:39:38.863312 (+ 661us) server_negotiation.cc:206] Beginning negotiation
0809 15:39:38.863325 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0809 15:39:38.863616 (+ 291us) server_negotiation.cc:407] Connection header received
0809 15:39:38.863702 (+ 86us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 15:39:38.863715 (+ 13us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 15:39:38.863927 (+ 212us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 15:39:38.864371 (+ 444us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 15:39:38.864667 (+ 296us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:39:38.865870 (+ 1203us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:39:38.867517 (+ 1647us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:39:38.868358 (+ 841us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:39:38.868468 (+ 110us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:39:38.868597 (+ 129us) server_negotiation.cc:1035] Waiting for connection context
0809 15:39:38.871081 (+ 2484us) server_negotiation.cc:299] Negotiation successful
0809 15:39:38.871182 (+ 101us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":564,"thread_start_us":166,"threads_started":1}
I20260809 15:39:38.871563 22258 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:39:38.862456 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.89.195:40017 (local address 127.0.0.1:40294)
0809 15:39:38.863262 (+ 806us) negotiation.cc:107] Waiting for socket to connect
0809 15:39:38.863311 (+ 49us) client_negotiation.cc:174] Beginning negotiation
0809 15:39:38.863477 (+ 166us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 15:39:38.864217 (+ 740us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 15:39:38.864223 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 15:39:38.864243 (+ 20us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 15:39:38.864557 (+ 314us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:39:38.864564 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:39:38.866001 (+ 1437us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:39:38.866004 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:39:38.867391 (+ 1387us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:39:38.867407 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:39:38.868665 (+ 1258us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:39:38.868669 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:39:38.868808 (+ 139us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:39:38.868830 (+ 22us) client_negotiation.cc:769] Sending connection context
0809 15:39:38.868897 (+ 67us) client_negotiation.cc:240] Negotiation successful
0809 15:39:38.868970 (+ 73us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":629,"mutex_wait_us":368,"spinlock_wait_cycles":4352,"thread_start_us":90,"threads_started":1}
I20260809 15:39:38.871824 22256 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:39:38.862732 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.89.193:39593 (local address 127.0.0.1:33496)
0809 15:39:38.864005 (+ 1273us) negotiation.cc:107] Waiting for socket to connect
0809 15:39:38.864022 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0809 15:39:38.864101 (+ 79us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 15:39:38.864832 (+ 731us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 15:39:38.864838 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 15:39:38.864856 (+ 18us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 15:39:38.864937 (+ 81us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:39:38.864944 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:39:38.866229 (+ 1285us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:39:38.866233 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:39:38.867474 (+ 1241us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:39:38.867484 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:39:38.868665 (+ 1181us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:39:38.868669 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:39:38.869093 (+ 424us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:39:38.869115 (+ 22us) client_negotiation.cc:769] Sending connection context
0809 15:39:38.869198 (+ 83us) client_negotiation.cc:240] Negotiation successful
0809 15:39:38.869264 (+ 66us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1188,"mutex_wait_us":46,"thread_start_us":91,"threads_started":1}
I20260809 15:39:38.871891 22023 tablet_service.cc:1467] Processing CreateTablet for tablet 0f3c9bea036a475f97ac0d6e089296bf (DEFAULT_TABLE table=test-table [id=67ef542c3458496cb5b235f164d1f5c7]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 15:39:38.872954 22023 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 0f3c9bea036a475f97ac0d6e089296bf. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 15:39:38.874150 22173 tablet_service.cc:1467] Processing CreateTablet for tablet 0f3c9bea036a475f97ac0d6e089296bf (DEFAULT_TABLE table=test-table [id=67ef542c3458496cb5b235f164d1f5c7]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 15:39:38.874867 21877 tablet_service.cc:1467] Processing CreateTablet for tablet 0f3c9bea036a475f97ac0d6e089296bf (DEFAULT_TABLE table=test-table [id=67ef542c3458496cb5b235f164d1f5c7]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 15:39:38.875293 22173 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 0f3c9bea036a475f97ac0d6e089296bf. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 15:39:38.875815 21877 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 0f3c9bea036a475f97ac0d6e089296bf. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 15:39:38.880606 22262 tablet_bootstrap.cc:492] T 0f3c9bea036a475f97ac0d6e089296bf P fe24e053bd2d42038b87676819285063: Bootstrap starting.
I20260809 15:39:38.882416 22264 tablet_bootstrap.cc:492] T 0f3c9bea036a475f97ac0d6e089296bf P 6f745da6a1a34eb7b394dec2ef2cd47c: Bootstrap starting.
I20260809 15:39:38.883215 22262 tablet_bootstrap.cc:654] T 0f3c9bea036a475f97ac0d6e089296bf P fe24e053bd2d42038b87676819285063: Neither blocks nor log segments found. Creating new log.
I20260809 15:39:38.883373 22263 tablet_bootstrap.cc:492] T 0f3c9bea036a475f97ac0d6e089296bf P 3b99c4280a9c48f5a293ad7e44e9ab5e: Bootstrap starting.
I20260809 15:39:38.884395 22262 log.cc:824] T 0f3c9bea036a475f97ac0d6e089296bf P fe24e053bd2d42038b87676819285063: Log is configured to *not* fsync() on all Append() calls
I20260809 15:39:38.884608 22264 tablet_bootstrap.cc:654] T 0f3c9bea036a475f97ac0d6e089296bf P 6f745da6a1a34eb7b394dec2ef2cd47c: Neither blocks nor log segments found. Creating new log.
I20260809 15:39:38.885838 22263 tablet_bootstrap.cc:654] T 0f3c9bea036a475f97ac0d6e089296bf P 3b99c4280a9c48f5a293ad7e44e9ab5e: Neither blocks nor log segments found. Creating new log.
I20260809 15:39:38.885860 22264 log.cc:824] T 0f3c9bea036a475f97ac0d6e089296bf P 6f745da6a1a34eb7b394dec2ef2cd47c: Log is configured to *not* fsync() on all Append() calls
I20260809 15:39:38.886332 22262 tablet_bootstrap.cc:492] T 0f3c9bea036a475f97ac0d6e089296bf P fe24e053bd2d42038b87676819285063: No bootstrap required, opened a new log
I20260809 15:39:38.886554 22262 ts_tablet_manager.cc:1397] T 0f3c9bea036a475f97ac0d6e089296bf P fe24e053bd2d42038b87676819285063: Time spent bootstrapping tablet: real 0.006s user 0.001s sys 0.003s
I20260809 15:39:38.886664 22263 log.cc:824] T 0f3c9bea036a475f97ac0d6e089296bf P 3b99c4280a9c48f5a293ad7e44e9ab5e: Log is configured to *not* fsync() on all Append() calls
I20260809 15:39:38.887913 22264 tablet_bootstrap.cc:492] T 0f3c9bea036a475f97ac0d6e089296bf P 6f745da6a1a34eb7b394dec2ef2cd47c: No bootstrap required, opened a new log
I20260809 15:39:38.888101 22264 ts_tablet_manager.cc:1397] T 0f3c9bea036a475f97ac0d6e089296bf P 6f745da6a1a34eb7b394dec2ef2cd47c: Time spent bootstrapping tablet: real 0.006s user 0.004s sys 0.000s
I20260809 15:39:38.888616 22263 tablet_bootstrap.cc:492] T 0f3c9bea036a475f97ac0d6e089296bf P 3b99c4280a9c48f5a293ad7e44e9ab5e: No bootstrap required, opened a new log
I20260809 15:39:38.888789 22263 ts_tablet_manager.cc:1397] T 0f3c9bea036a475f97ac0d6e089296bf P 3b99c4280a9c48f5a293ad7e44e9ab5e: Time spent bootstrapping tablet: real 0.006s user 0.000s sys 0.004s
I20260809 15:39:38.890395 22262 raft_consensus.cc:348] T 0f3c9bea036a475f97ac0d6e089296bf P fe24e053bd2d42038b87676819285063 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3b99c4280a9c48f5a293ad7e44e9ab5e" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 40017 } } peers { permanent_uuid: "fe24e053bd2d42038b87676819285063" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 46783 } } peers { permanent_uuid: "6f745da6a1a34eb7b394dec2ef2cd47c" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 39593 } }
I20260809 15:39:38.890671 22262 raft_consensus.cc:374] T 0f3c9bea036a475f97ac0d6e089296bf P fe24e053bd2d42038b87676819285063 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 15:39:38.890733 22262 raft_consensus.cc:729] T 0f3c9bea036a475f97ac0d6e089296bf P fe24e053bd2d42038b87676819285063 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fe24e053bd2d42038b87676819285063, State: Initialized, Role: FOLLOWER
I20260809 15:39:38.891251 22262 consensus_queue.cc:260] T 0f3c9bea036a475f97ac0d6e089296bf P fe24e053bd2d42038b87676819285063 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3b99c4280a9c48f5a293ad7e44e9ab5e" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 40017 } } peers { permanent_uuid: "fe24e053bd2d42038b87676819285063" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 46783 } } peers { permanent_uuid: "6f745da6a1a34eb7b394dec2ef2cd47c" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 39593 } }
I20260809 15:39:38.891906 22095 heartbeater.cc:503] Master 127.3.89.254:42961 was elected leader, sending a full tablet report...
I20260809 15:39:38.891906 22264 raft_consensus.cc:348] T 0f3c9bea036a475f97ac0d6e089296bf P 6f745da6a1a34eb7b394dec2ef2cd47c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3b99c4280a9c48f5a293ad7e44e9ab5e" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 40017 } } peers { permanent_uuid: "fe24e053bd2d42038b87676819285063" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 46783 } } peers { permanent_uuid: "6f745da6a1a34eb7b394dec2ef2cd47c" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 39593 } }
I20260809 15:39:38.892213 22264 raft_consensus.cc:374] T 0f3c9bea036a475f97ac0d6e089296bf P 6f745da6a1a34eb7b394dec2ef2cd47c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 15:39:38.892277 22264 raft_consensus.cc:729] T 0f3c9bea036a475f97ac0d6e089296bf P 6f745da6a1a34eb7b394dec2ef2cd47c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6f745da6a1a34eb7b394dec2ef2cd47c, State: Initialized, Role: FOLLOWER
I20260809 15:39:38.892544 22263 raft_consensus.cc:348] T 0f3c9bea036a475f97ac0d6e089296bf P 3b99c4280a9c48f5a293ad7e44e9ab5e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3b99c4280a9c48f5a293ad7e44e9ab5e" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 40017 } } peers { permanent_uuid: "fe24e053bd2d42038b87676819285063" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 46783 } } peers { permanent_uuid: "6f745da6a1a34eb7b394dec2ef2cd47c" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 39593 } }
I20260809 15:39:38.892695 22264 consensus_queue.cc:260] T 0f3c9bea036a475f97ac0d6e089296bf P 6f745da6a1a34eb7b394dec2ef2cd47c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3b99c4280a9c48f5a293ad7e44e9ab5e" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 40017 } } peers { permanent_uuid: "fe24e053bd2d42038b87676819285063" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 46783 } } peers { permanent_uuid: "6f745da6a1a34eb7b394dec2ef2cd47c" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 39593 } }
I20260809 15:39:38.892836 22263 raft_consensus.cc:374] T 0f3c9bea036a475f97ac0d6e089296bf P 3b99c4280a9c48f5a293ad7e44e9ab5e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 15:39:38.892884 22263 raft_consensus.cc:729] T 0f3c9bea036a475f97ac0d6e089296bf P 3b99c4280a9c48f5a293ad7e44e9ab5e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3b99c4280a9c48f5a293ad7e44e9ab5e, State: Initialized, Role: FOLLOWER
I20260809 15:39:38.893356 22263 consensus_queue.cc:260] T 0f3c9bea036a475f97ac0d6e089296bf P 3b99c4280a9c48f5a293ad7e44e9ab5e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3b99c4280a9c48f5a293ad7e44e9ab5e" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 40017 } } peers { permanent_uuid: "fe24e053bd2d42038b87676819285063" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 46783 } } peers { permanent_uuid: "6f745da6a1a34eb7b394dec2ef2cd47c" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 39593 } }
I20260809 15:39:38.893714 22264 ts_tablet_manager.cc:1428] T 0f3c9bea036a475f97ac0d6e089296bf P 6f745da6a1a34eb7b394dec2ef2cd47c: Time spent starting tablet: real 0.006s user 0.006s sys 0.000s
I20260809 15:39:38.894141 21948 heartbeater.cc:503] Master 127.3.89.254:42961 was elected leader, sending a full tablet report...
I20260809 15:39:38.894294 22263 ts_tablet_manager.cc:1428] T 0f3c9bea036a475f97ac0d6e089296bf P 3b99c4280a9c48f5a293ad7e44e9ab5e: Time spent starting tablet: real 0.005s user 0.005s sys 0.001s
I20260809 15:39:38.894989 22244 heartbeater.cc:503] Master 127.3.89.254:42961 was elected leader, sending a full tablet report...
I20260809 15:39:38.897610 22262 ts_tablet_manager.cc:1428] T 0f3c9bea036a475f97ac0d6e089296bf P fe24e053bd2d42038b87676819285063: Time spent starting tablet: real 0.011s user 0.003s sys 0.004s
I20260809 15:39:39.006699 22268 raft_consensus.cc:482] T 0f3c9bea036a475f97ac0d6e089296bf P fe24e053bd2d42038b87676819285063 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 15:39:39.006939 22268 raft_consensus.cc:504] T 0f3c9bea036a475f97ac0d6e089296bf P fe24e053bd2d42038b87676819285063 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3b99c4280a9c48f5a293ad7e44e9ab5e" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 40017 } } peers { permanent_uuid: "fe24e053bd2d42038b87676819285063" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 46783 } } peers { permanent_uuid: "6f745da6a1a34eb7b394dec2ef2cd47c" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 39593 } }
I20260809 15:39:39.008204 22268 leader_election.cc:288] T 0f3c9bea036a475f97ac0d6e089296bf P fe24e053bd2d42038b87676819285063 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 3b99c4280a9c48f5a293ad7e44e9ab5e (127.3.89.195:40017), 6f745da6a1a34eb7b394dec2ef2cd47c (127.3.89.193:39593)
I20260809 15:39:39.012316 22098 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:39:39.008467 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.89.193:39593 (local address 127.3.89.194:49341)
0809 15:39:39.008604 (+ 137us) negotiation.cc:107] Waiting for socket to connect
0809 15:39:39.008622 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0809 15:39:39.008720 (+ 98us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 15:39:39.009026 (+ 306us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 15:39:39.009029 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 15:39:39.009047 (+ 18us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 15:39:39.009196 (+ 149us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:39:39.009205 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:39:39.010138 (+ 933us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:39:39.010142 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:39:39.011106 (+ 964us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:39:39.011115 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:39:39.011876 (+ 761us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:39:39.011880 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:39:39.011987 (+ 107us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:39:39.012024 (+ 37us) client_negotiation.cc:769] Sending connection context
0809 15:39:39.012105 (+ 81us) client_negotiation.cc:240] Negotiation successful
0809 15:39:39.012182 (+ 77us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":48,"mutex_wait_us":1}
I20260809 15:39:39.012360 22261 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:39:39.008582 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.89.194:49341 (local address 127.3.89.193:39593)
0809 15:39:39.008709 (+ 127us) server_negotiation.cc:206] Beginning negotiation
0809 15:39:39.008712 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0809 15:39:39.008727 (+ 15us) server_negotiation.cc:407] Connection header received
0809 15:39:39.008872 (+ 145us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 15:39:39.008875 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 15:39:39.008921 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 15:39:39.009011 (+ 90us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 15:39:39.009358 (+ 347us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:39:39.010016 (+ 658us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:39:39.011229 (+ 1213us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:39:39.011716 (+ 487us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:39:39.011808 (+ 92us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:39:39.011849 (+ 41us) server_negotiation.cc:1035] Waiting for connection context
0809 15:39:39.012131 (+ 282us) server_negotiation.cc:299] Negotiation successful
0809 15:39:39.012207 (+ 76us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":59}
I20260809 15:39:39.012532 22259 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:39:39.008551 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.89.194:45521 (local address 127.3.89.195:40017)
0809 15:39:39.008692 (+ 141us) server_negotiation.cc:206] Beginning negotiation
0809 15:39:39.008697 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 15:39:39.009185 (+ 488us) server_negotiation.cc:407] Connection header received
0809 15:39:39.009293 (+ 108us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 15:39:39.009298 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 15:39:39.009366 (+ 68us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 15:39:39.009425 (+ 59us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 15:39:39.009734 (+ 309us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:39:39.010308 (+ 574us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:39:39.011487 (+ 1179us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:39:39.011963 (+ 476us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:39:39.012093 (+ 130us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:39:39.012142 (+ 49us) server_negotiation.cc:1035] Waiting for connection context
0809 15:39:39.012352 (+ 210us) server_negotiation.cc:299] Negotiation successful
0809 15:39:39.012429 (+ 77us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":51}
I20260809 15:39:39.012774 22271 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:39:39.008469 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.89.195:40017 (local address 127.3.89.194:45521)
0809 15:39:39.009086 (+ 617us) negotiation.cc:107] Waiting for socket to connect
0809 15:39:39.009098 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0809 15:39:39.009188 (+ 90us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 15:39:39.009477 (+ 289us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 15:39:39.009480 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 15:39:39.009498 (+ 18us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 15:39:39.009624 (+ 126us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:39:39.009631 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:39:39.010420 (+ 789us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:39:39.010422 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:39:39.011360 (+ 938us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:39:39.011370 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:39:39.012107 (+ 737us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:39:39.012112 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:39:39.012215 (+ 103us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:39:39.012237 (+ 22us) client_negotiation.cc:769] Sending connection context
0809 15:39:39.012316 (+ 79us) client_negotiation.cc:240] Negotiation successful
0809 15:39:39.012544 (+ 228us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":552,"mutex_wait_us":287,"thread_start_us":132,"threads_started":1}
I20260809 15:39:39.013206 21899 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0f3c9bea036a475f97ac0d6e089296bf" candidate_uuid: "fe24e053bd2d42038b87676819285063" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6f745da6a1a34eb7b394dec2ef2cd47c" is_pre_election: true
I20260809 15:39:39.013446 22194 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0f3c9bea036a475f97ac0d6e089296bf" candidate_uuid: "fe24e053bd2d42038b87676819285063" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3b99c4280a9c48f5a293ad7e44e9ab5e" is_pre_election: true
I20260809 15:39:39.013561 21899 raft_consensus.cc:2454] T 0f3c9bea036a475f97ac0d6e089296bf P 6f745da6a1a34eb7b394dec2ef2cd47c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fe24e053bd2d42038b87676819285063 in term 0.
I20260809 15:39:39.013702 22194 raft_consensus.cc:2454] T 0f3c9bea036a475f97ac0d6e089296bf P 3b99c4280a9c48f5a293ad7e44e9ab5e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fe24e053bd2d42038b87676819285063 in term 0.
I20260809 15:39:39.014112 21971 leader_election.cc:302] T 0f3c9bea036a475f97ac0d6e089296bf P fe24e053bd2d42038b87676819285063 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6f745da6a1a34eb7b394dec2ef2cd47c, fe24e053bd2d42038b87676819285063; no voters:
I20260809 15:39:39.014400 22268 raft_consensus.cc:2781] T 0f3c9bea036a475f97ac0d6e089296bf P fe24e053bd2d42038b87676819285063 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260809 15:39:39.014489 22268 raft_consensus.cc:482] T 0f3c9bea036a475f97ac0d6e089296bf P fe24e053bd2d42038b87676819285063 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 15:39:39.014521 22268 raft_consensus.cc:3037] T 0f3c9bea036a475f97ac0d6e089296bf P fe24e053bd2d42038b87676819285063 [term 0 FOLLOWER]: Advancing to term 1
I20260809 15:39:39.015652 22268 raft_consensus.cc:504] T 0f3c9bea036a475f97ac0d6e089296bf P fe24e053bd2d42038b87676819285063 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3b99c4280a9c48f5a293ad7e44e9ab5e" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 40017 } } peers { permanent_uuid: "fe24e053bd2d42038b87676819285063" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 46783 } } peers { permanent_uuid: "6f745da6a1a34eb7b394dec2ef2cd47c" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 39593 } }
I20260809 15:39:39.016047 22268 leader_election.cc:288] T 0f3c9bea036a475f97ac0d6e089296bf P fe24e053bd2d42038b87676819285063 [CANDIDATE]: Term 1 election: Requested vote from peers 3b99c4280a9c48f5a293ad7e44e9ab5e (127.3.89.195:40017), 6f745da6a1a34eb7b394dec2ef2cd47c (127.3.89.193:39593)
I20260809 15:39:39.016368 22194 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0f3c9bea036a475f97ac0d6e089296bf" candidate_uuid: "fe24e053bd2d42038b87676819285063" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3b99c4280a9c48f5a293ad7e44e9ab5e"
I20260809 15:39:39.016487 22194 raft_consensus.cc:3037] T 0f3c9bea036a475f97ac0d6e089296bf P 3b99c4280a9c48f5a293ad7e44e9ab5e [term 0 FOLLOWER]: Advancing to term 1
I20260809 15:39:39.016530 21899 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0f3c9bea036a475f97ac0d6e089296bf" candidate_uuid: "fe24e053bd2d42038b87676819285063" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6f745da6a1a34eb7b394dec2ef2cd47c"
I20260809 15:39:39.016693 21899 raft_consensus.cc:3037] T 0f3c9bea036a475f97ac0d6e089296bf P 6f745da6a1a34eb7b394dec2ef2cd47c [term 0 FOLLOWER]: Advancing to term 1
I20260809 15:39:39.017627 22194 raft_consensus.cc:2454] T 0f3c9bea036a475f97ac0d6e089296bf P 3b99c4280a9c48f5a293ad7e44e9ab5e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fe24e053bd2d42038b87676819285063 in term 1.
I20260809 15:39:39.017943 21970 leader_election.cc:302] T 0f3c9bea036a475f97ac0d6e089296bf P fe24e053bd2d42038b87676819285063 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3b99c4280a9c48f5a293ad7e44e9ab5e, fe24e053bd2d42038b87676819285063; no voters:
I20260809 15:39:39.017953 21899 raft_consensus.cc:2454] T 0f3c9bea036a475f97ac0d6e089296bf P 6f745da6a1a34eb7b394dec2ef2cd47c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fe24e053bd2d42038b87676819285063 in term 1.
I20260809 15:39:39.018112 22268 raft_consensus.cc:2781] T 0f3c9bea036a475f97ac0d6e089296bf P fe24e053bd2d42038b87676819285063 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 15:39:39.018324 22268 raft_consensus.cc:686] T 0f3c9bea036a475f97ac0d6e089296bf P fe24e053bd2d42038b87676819285063 [term 1 LEADER]: Becoming Leader. State: Replica: fe24e053bd2d42038b87676819285063, State: Running, Role: LEADER
I20260809 15:39:39.018692 22268 consensus_queue.cc:237] T 0f3c9bea036a475f97ac0d6e089296bf P fe24e053bd2d42038b87676819285063 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3b99c4280a9c48f5a293ad7e44e9ab5e" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 40017 } } peers { permanent_uuid: "fe24e053bd2d42038b87676819285063" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 46783 } } peers { permanent_uuid: "6f745da6a1a34eb7b394dec2ef2cd47c" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 39593 } }
I20260809 15:39:39.021903 21744 catalog_manager.cc:5495] T 0f3c9bea036a475f97ac0d6e089296bf P fe24e053bd2d42038b87676819285063 reported cstate change: term changed from 0 to 1, leader changed from <none> to fe24e053bd2d42038b87676819285063 (127.3.89.194). New cstate: current_term: 1 leader_uuid: "fe24e053bd2d42038b87676819285063" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3b99c4280a9c48f5a293ad7e44e9ab5e" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 40017 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "fe24e053bd2d42038b87676819285063" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 46783 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6f745da6a1a34eb7b394dec2ef2cd47c" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 39593 } health_report { overall_health: UNKNOWN } } }
I20260809 15:39:39.063156 22260 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:39:39.059962 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41860 (local address 127.3.89.194:46783)
0809 15:39:39.060084 (+ 122us) server_negotiation.cc:206] Beginning negotiation
0809 15:39:39.060088 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 15:39:39.060101 (+ 13us) server_negotiation.cc:407] Connection header received
0809 15:39:39.060193 (+ 92us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 15:39:39.060196 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 15:39:39.060253 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 15:39:39.060355 (+ 102us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 15:39:39.060572 (+ 217us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:39:39.061057 (+ 485us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:39:39.061841 (+ 784us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:39:39.062041 (+ 200us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:39:39.062115 (+ 74us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:39:39.062379 (+ 264us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 15:39:39.062780 (+ 401us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 15:39:39.062871 (+ 91us) server_negotiation.cc:1035] Waiting for connection context
0809 15:39:39.062928 (+ 57us) server_negotiation.cc:299] Negotiation successful
0809 15:39:39.063031 (+ 103us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":44}
W20260809 15:39:39.068166 22245 tablet.cc:2369] T 0f3c9bea036a475f97ac0d6e089296bf P 3b99c4280a9c48f5a293ad7e44e9ab5e: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260809 15:39:39.068760 22194 raft_consensus.cc:1261] T 0f3c9bea036a475f97ac0d6e089296bf P 3b99c4280a9c48f5a293ad7e44e9ab5e [term 1 FOLLOWER]: Refusing update from remote peer fe24e053bd2d42038b87676819285063: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 15:39:39.068761 21899 raft_consensus.cc:1261] T 0f3c9bea036a475f97ac0d6e089296bf P 6f745da6a1a34eb7b394dec2ef2cd47c [term 1 FOLLOWER]: Refusing update from remote peer fe24e053bd2d42038b87676819285063: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 15:39:39.069470 22268 consensus_queue.cc:1035] T 0f3c9bea036a475f97ac0d6e089296bf P fe24e053bd2d42038b87676819285063 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6f745da6a1a34eb7b394dec2ef2cd47c" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 39593 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 15:39:39.069629 22272 consensus_queue.cc:1035] T 0f3c9bea036a475f97ac0d6e089296bf P fe24e053bd2d42038b87676819285063 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3b99c4280a9c48f5a293ad7e44e9ab5e" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 40017 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 15:39:39.076830 22275 mvcc.cc:205] Tried to move back new op lower bound from 7316643754258739200 to 7316643754063364096. Current Snapshot: MvccSnapshot[applied={T|T < 7316643754258739200}]
I20260809 15:39:39.077664 22276 mvcc.cc:205] Tried to move back new op lower bound from 7316643754258739200 to 7316643754063364096. Current Snapshot: MvccSnapshot[applied={T|T < 7316643754258739200}]
I20260809 15:39:39.081604 22278 mvcc.cc:205] Tried to move back new op lower bound from 7316643754258739200 to 7316643754063364096. Current Snapshot: MvccSnapshot[applied={T|T < 7316643754258739200 or (T in {7316643754258739200})}]
I20260809 15:39:39.085086 21744 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:49724:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 15:39:39.085222 21744 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:49724:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 15:39:39.087657 21744 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 1ee6dacea0274b3e9315e544bd186631: Sending DeleteTablet for 3 replicas of tablet 0f3c9bea036a475f97ac0d6e089296bf
I20260809 15:39:39.088475 22173 tablet_service.cc:1514] Processing DeleteTablet for tablet 0f3c9bea036a475f97ac0d6e089296bf with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 15:39:39 UTC) from {username='kudu', principal='kudu/127.3.89.254@KRBTEST.COM'} at 127.0.0.1:40294
I20260809 15:39:39.088477 21877 tablet_service.cc:1514] Processing DeleteTablet for tablet 0f3c9bea036a475f97ac0d6e089296bf with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 15:39:39 UTC) from {username='kudu', principal='kudu/127.3.89.254@KRBTEST.COM'} at 127.0.0.1:33496
I20260809 15:39:39.088627 22023 tablet_service.cc:1514] Processing DeleteTablet for tablet 0f3c9bea036a475f97ac0d6e089296bf with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 15:39:39 UTC) from {username='kudu', principal='kudu/127.3.89.254@KRBTEST.COM'} at 127.0.0.1:41854
I20260809 15:39:39.088956 22286 tablet_replica.cc:330] T 0f3c9bea036a475f97ac0d6e089296bf P 6f745da6a1a34eb7b394dec2ef2cd47c: stopping tablet replica
I20260809 15:39:39.089032 22287 tablet_replica.cc:330] T 0f3c9bea036a475f97ac0d6e089296bf P fe24e053bd2d42038b87676819285063: stopping tablet replica
I20260809 15:39:39.089258 22286 raft_consensus.cc:2229] T 0f3c9bea036a475f97ac0d6e089296bf P 6f745da6a1a34eb7b394dec2ef2cd47c [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 15:39:39.089519 22285 tablet_replica.cc:330] T 0f3c9bea036a475f97ac0d6e089296bf P 3b99c4280a9c48f5a293ad7e44e9ab5e: stopping tablet replica
I20260809 15:39:39.089537 22286 raft_consensus.cc:2258] T 0f3c9bea036a475f97ac0d6e089296bf P 6f745da6a1a34eb7b394dec2ef2cd47c [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 15:39:39.089838 3431 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskm314YR/build/debug/bin/kudu with pid 21808
I20260809 15:39:39.091537 22285 raft_consensus.cc:2229] T 0f3c9bea036a475f97ac0d6e089296bf P 3b99c4280a9c48f5a293ad7e44e9ab5e [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 15:39:39.091869 22285 raft_consensus.cc:2258] T 0f3c9bea036a475f97ac0d6e089296bf P 3b99c4280a9c48f5a293ad7e44e9ab5e [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 15:39:39.093786 22285 ts_tablet_manager.cc:1905] T 0f3c9bea036a475f97ac0d6e089296bf P 3b99c4280a9c48f5a293ad7e44e9ab5e: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 15:39:39.096647 22285 ts_tablet_manager.cc:1918] T 0f3c9bea036a475f97ac0d6e089296bf P 3b99c4280a9c48f5a293ad7e44e9ab5e: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 15:39:39.096804 22285 log.cc:1196] T 0f3c9bea036a475f97ac0d6e089296bf P 3b99c4280a9c48f5a293ad7e44e9ab5e: Deleting WAL directory at /tmp/dist-test-taskm314YR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786289732823074-3431-0/minicluster-data/ts-2/wal/wals/0f3c9bea036a475f97ac0d6e089296bf
I20260809 15:39:39.097232 22285 ts_tablet_manager.cc:1939] T 0f3c9bea036a475f97ac0d6e089296bf P 3b99c4280a9c48f5a293ad7e44e9ab5e: Deleting consensus metadata
I20260809 15:39:39.098516 21728 catalog_manager.cc:4841] TS 3b99c4280a9c48f5a293ad7e44e9ab5e (127.3.89.195:40017): tablet 0f3c9bea036a475f97ac0d6e089296bf (table test-table [id=67ef542c3458496cb5b235f164d1f5c7]) successfully deleted
I20260809 15:39:39.089267 22287 raft_consensus.cc:2229] T 0f3c9bea036a475f97ac0d6e089296bf P fe24e053bd2d42038b87676819285063 [term 1 LEADER]: Raft consensus shutting down.
I20260809 15:39:39.100641 22287 raft_consensus.cc:2258] T 0f3c9bea036a475f97ac0d6e089296bf P fe24e053bd2d42038b87676819285063 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 15:39:39.102167 22287 ts_tablet_manager.cc:1905] T 0f3c9bea036a475f97ac0d6e089296bf P fe24e053bd2d42038b87676819285063: Deleting tablet data with delete state TABLET_DATA_DELETED
W20260809 15:39:39.105119 21729 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.3.89.193:39593 (error 108)
I20260809 15:39:39.105468 22287 ts_tablet_manager.cc:1918] T 0f3c9bea036a475f97ac0d6e089296bf P fe24e053bd2d42038b87676819285063: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 15:39:39.105566 22287 log.cc:1196] T 0f3c9bea036a475f97ac0d6e089296bf P fe24e053bd2d42038b87676819285063: Deleting WAL directory at /tmp/dist-test-taskm314YR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786289732823074-3431-0/minicluster-data/ts-1/wal/wals/0f3c9bea036a475f97ac0d6e089296bf
I20260809 15:39:39.105899 22287 ts_tablet_manager.cc:1939] T 0f3c9bea036a475f97ac0d6e089296bf P fe24e053bd2d42038b87676819285063: Deleting consensus metadata
I20260809 15:39:39.106027 3431 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskm314YR/build/debug/bin/kudu with pid 21953
I20260809 15:39:39.107124 22256 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:39:39.106338 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.89.193:39593 (local address 127.0.0.1:33510)
0809 15:39:39.106891 (+ 553us) negotiation.cc:107] Waiting for socket to connect
0809 15:39:39.107020 (+ 129us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.89.193:39593: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":473}
W20260809 15:39:39.107314 21729 catalog_manager.cc:4584] TS 6f745da6a1a34eb7b394dec2ef2cd47c (127.3.89.193:39593): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 0f3c9bea036a475f97ac0d6e089296bf: Network error: Client connection negotiation failed: client connection to 127.3.89.193:39593: connect: Connection refused (error 111)
I20260809 15:39:39.112905 3431 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskm314YR/build/debug/bin/kudu with pid 22101
I20260809 15:39:39.113229 22256 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:39:39.112924 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.89.194:46783 (local address 127.0.0.1:41866)
0809 15:39:39.113076 (+ 152us) negotiation.cc:107] Waiting for socket to connect
0809 15:39:39.113140 (+ 64us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.89.194:46783: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":57}
I20260809 15:39:39.119360 3431 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskm314YR/build/debug/bin/kudu with pid 21709
2026-08-09T15:39:39Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/0 (2372 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/1
2026-08-09T15:39:39Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T15:39:39Z Disabled control of system clock
I20260809 15:39:39.151357 3431 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskm314YR/build/debug/bin/kudu
/tmp/dist-test-taskm314YR/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786289732823074-3431-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786289732823074-3431-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786289732823074-3431-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786289732823074-3431-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.3.89.254:44325
--webserver_interface=127.3.89.254
--webserver_port=0
--builtin_ntp_servers=127.3.89.212:46353
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.3.89.254:44325
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260809 15:39:39.260193 22294 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 15:39:39.260426 22294 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 15:39:39.260460 22294 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 15:39:39.264389 22294 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 15:39:39.264439 22294 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 15:39:39.264456 22294 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 15:39:39.264477 22294 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 15:39:39.264535 22294 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 15:39:39.269223 22294 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.3.89.212:46353
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786289732823074-3431-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786289732823074-3431-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.3.89.254:44325
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.3.89.254:44325
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786289732823074-3431-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.3.89.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.22294
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786289732823074-3431-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 09 Aug 2026 14:46:55 UTC on ccycloud.quasar-yswkmf.root.comops.site
build id 4576540
I20260809 15:39:39.270190 22294 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 15:39:39.271495 22294 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 15:39:39.279481 22299 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 15:39:39.279448 22294 server_base.cc:1034] running on GCE node
W20260809 15:39:39.279404 22302 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 15:39:39.279412 22300 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 15:39:39.280201 22294 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 15:39:39.281193 22294 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 15:39:39.282383 22294 hybrid_clock.cc:648] HybridClock initialized: now 1786289979282372 us; error 25 us; skew 500 ppm
I20260809 15:39:39.284200 22294 webserver.cc:460] Webserver started at http://127.3.89.254:42199/ using document root <none> and password file <none>
I20260809 15:39:39.284739 22294 fs_manager.cc:359] Metadata directory not provided
I20260809 15:39:39.284833 22294 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 15:39:39.285053 22294 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 15:39:39.287046 22294 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskm314YR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786289732823074-3431-0/minicluster-data/master-0/data/instance:
uuid: "2e607fffbe0543dc8b5b928060542e4e"
format_stamp: "Formatted at 2026-08-09 15:39:39 on dist-test-slave-2w3w"
server_key: "68c3ddd5fcdf7ea6698fd9d2555dfe78"
server_key_iv: "7dafe67ddd262df29c29f24bb092d0da"
server_key_version: "encryptionkey@0"
I20260809 15:39:39.287617 22294 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskm314YR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786289732823074-3431-0/minicluster-data/master-0/wal/instance:
uuid: "2e607fffbe0543dc8b5b928060542e4e"
format_stamp: "Formatted at 2026-08-09 15:39:39 on dist-test-slave-2w3w"
server_key: "68c3ddd5fcdf7ea6698fd9d2555dfe78"
server_key_iv: "7dafe67ddd262df29c29f24bb092d0da"
server_key_version: "encryptionkey@0"
I20260809 15:39:39.291244 22294 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.003s sys 0.001s
I20260809 15:39:39.293540 22308 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 15:39:39.294687 22294 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260809 15:39:39.294838 22294 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskm314YR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786289732823074-3431-0/minicluster-data/master-0/data,/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786289732823074-3431-0/minicluster-data/master-0/wal
uuid: "2e607fffbe0543dc8b5b928060542e4e"
format_stamp: "Formatted at 2026-08-09 15:39:39 on dist-test-slave-2w3w"
server_key: "68c3ddd5fcdf7ea6698fd9d2555dfe78"
server_key_iv: "7dafe67ddd262df29c29f24bb092d0da"
server_key_version: "encryptionkey@0"
I20260809 15:39:39.294943 22294 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskm314YR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786289732823074-3431-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskm314YR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786289732823074-3431-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskm314YR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786289732823074-3431-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 15:39:39.314604 22294 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 15:39:39.315269 22294 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 15:39:39.315464 22294 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 15:39:39.322273 22294 rpc_server.cc:307] RPC server started. Bound to: 127.3.89.254:44325
I20260809 15:39:39.322285 22364 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.3.89.254:44325 every 8 connection(s)
I20260809 15:39:39.323228 22294 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskm314YR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786289732823074-3431-0/minicluster-data/master-0/data/info.pb
I20260809 15:39:39.326140 22365 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 15:39:39.327126 3431 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskm314YR/build/debug/bin/kudu as pid 22294
I20260809 15:39:39.327258 3431 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskm314YR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786289732823074-3431-0/minicluster-data/master-0/wal/instance
I20260809 15:39:39.327612 3431 external_mini_cluster.cc:1428] Setting key 42e9f7ffd6f5548c43a5f3f87f77d452
I20260809 15:39:39.332226 22365 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2e607fffbe0543dc8b5b928060542e4e: Bootstrap starting.
I20260809 15:39:39.334481 22365 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 2e607fffbe0543dc8b5b928060542e4e: Neither blocks nor log segments found. Creating new log.
I20260809 15:39:39.335278 22365 log.cc:824] T 00000000000000000000000000000000 P 2e607fffbe0543dc8b5b928060542e4e: Log is configured to *not* fsync() on all Append() calls
I20260809 15:39:39.336911 22369 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:39:39.328990 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53838 (local address 127.3.89.254:44325)
0809 15:39:39.329600 (+ 610us) server_negotiation.cc:206] Beginning negotiation
0809 15:39:39.329614 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0809 15:39:39.329647 (+ 33us) server_negotiation.cc:407] Connection header received
0809 15:39:39.330270 (+ 623us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 15:39:39.330296 (+ 26us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 15:39:39.330656 (+ 360us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 15:39:39.330973 (+ 317us) server_negotiation.cc:226] Negotiated authn=SASL
0809 15:39:39.331414 (+ 441us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:39:39.332394 (+ 980us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:39:39.333279 (+ 885us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:39:39.333741 (+ 462us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:39:39.333936 (+ 195us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:39:39.334920 (+ 984us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 15:39:39.334924 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 15:39:39.334936 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 15:39:39.335398 (+ 462us) server_negotiation.cc:922] calling sasl_server_start()
0809 15:39:39.335464 (+ 66us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 15:39:39.335485 (+ 21us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 15:39:39.335594 (+ 109us) server_negotiation.cc:1035] Waiting for connection context
0809 15:39:39.335748 (+ 154us) server_negotiation.cc:299] Negotiation successful
0809 15:39:39.336025 (+ 277us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":296,"thread_start_us":137,"threads_started":1}
I20260809 15:39:39.337394 22365 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2e607fffbe0543dc8b5b928060542e4e: No bootstrap required, opened a new log
I20260809 15:39:39.340968 22365 raft_consensus.cc:348] T 00000000000000000000000000000000 P 2e607fffbe0543dc8b5b928060542e4e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2e607fffbe0543dc8b5b928060542e4e" member_type: VOTER last_known_addr { host: "127.3.89.254" port: 44325 } }
I20260809 15:39:39.341167 22365 raft_consensus.cc:374] T 00000000000000000000000000000000 P 2e607fffbe0543dc8b5b928060542e4e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 15:39:39.341200 22365 raft_consensus.cc:729] T 00000000000000000000000000000000 P 2e607fffbe0543dc8b5b928060542e4e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2e607fffbe0543dc8b5b928060542e4e, State: Initialized, Role: FOLLOWER
I20260809 15:39:39.341666 22365 consensus_queue.cc:260] T 00000000000000000000000000000000 P 2e607fffbe0543dc8b5b928060542e4e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2e607fffbe0543dc8b5b928060542e4e" member_type: VOTER last_known_addr { host: "127.3.89.254" port: 44325 } }
I20260809 15:39:39.341859 22365 raft_consensus.cc:388] T 00000000000000000000000000000000 P 2e607fffbe0543dc8b5b928060542e4e [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 15:39:39.341935 22365 raft_consensus.cc:482] T 00000000000000000000000000000000 P 2e607fffbe0543dc8b5b928060542e4e [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 15:39:39.342038 22365 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 2e607fffbe0543dc8b5b928060542e4e [term 0 FOLLOWER]: Advancing to term 1
I20260809 15:39:39.342901 22365 raft_consensus.cc:504] T 00000000000000000000000000000000 P 2e607fffbe0543dc8b5b928060542e4e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2e607fffbe0543dc8b5b928060542e4e" member_type: VOTER last_known_addr { host: "127.3.89.254" port: 44325 } }
I20260809 15:39:39.343262 22365 leader_election.cc:302] T 00000000000000000000000000000000 P 2e607fffbe0543dc8b5b928060542e4e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 2e607fffbe0543dc8b5b928060542e4e; no voters:
I20260809 15:39:39.343562 22365 leader_election.cc:288] T 00000000000000000000000000000000 P 2e607fffbe0543dc8b5b928060542e4e [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 15:39:39.343677 22372 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 2e607fffbe0543dc8b5b928060542e4e [term 1 FOLLOWER]: Leader election won for term 1
I20260809 15:39:39.343935 22372 raft_consensus.cc:686] T 00000000000000000000000000000000 P 2e607fffbe0543dc8b5b928060542e4e [term 1 LEADER]: Becoming Leader. State: Replica: 2e607fffbe0543dc8b5b928060542e4e, State: Running, Role: LEADER
I20260809 15:39:39.344249 22372 consensus_queue.cc:237] T 00000000000000000000000000000000 P 2e607fffbe0543dc8b5b928060542e4e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2e607fffbe0543dc8b5b928060542e4e" member_type: VOTER last_known_addr { host: "127.3.89.254" port: 44325 } }
I20260809 15:39:39.344820 22365 sys_catalog.cc:564] T 00000000000000000000000000000000 P 2e607fffbe0543dc8b5b928060542e4e [sys.catalog]: configured and running, proceeding with master startup.
I20260809 15:39:39.345927 22374 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2e607fffbe0543dc8b5b928060542e4e [sys.catalog]: SysCatalogTable state changed. Reason: New leader 2e607fffbe0543dc8b5b928060542e4e. Latest consensus state: current_term: 1 leader_uuid: "2e607fffbe0543dc8b5b928060542e4e" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2e607fffbe0543dc8b5b928060542e4e" member_type: VOTER last_known_addr { host: "127.3.89.254" port: 44325 } } }
I20260809 15:39:39.346071 22374 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2e607fffbe0543dc8b5b928060542e4e [sys.catalog]: This master's current role is: LEADER
I20260809 15:39:39.346371 22373 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2e607fffbe0543dc8b5b928060542e4e [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "2e607fffbe0543dc8b5b928060542e4e" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2e607fffbe0543dc8b5b928060542e4e" member_type: VOTER last_known_addr { host: "127.3.89.254" port: 44325 } } }
I20260809 15:39:39.346434 22381 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 15:39:39.346483 22373 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2e607fffbe0543dc8b5b928060542e4e [sys.catalog]: This master's current role is: LEADER
I20260809 15:39:39.349553 22381 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 15:39:39.355082 22381 catalog_manager.cc:1347] Generated new cluster ID: 6ea1263d66fd497ea8863d0edf862ff4
I20260809 15:39:39.355156 22381 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 15:39:39.393923 22381 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 15:39:39.395190 22381 catalog_manager.cc:1504] Loading token signing keys...
I20260809 15:39:39.399972 22381 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 2e607fffbe0543dc8b5b928060542e4e: Generated new TSK 0
I20260809 15:39:39.400609 22381 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260809 15:39:39.407155 3431 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskm314YR/build/debug/bin/kudu
/tmp/dist-test-taskm314YR/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786289732823074-3431-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786289732823074-3431-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786289732823074-3431-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786289732823074-3431-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.3.89.193:0
--local_ip_for_outbound_sockets=127.3.89.193
--webserver_interface=127.3.89.193
--webserver_port=0
--tserver_master_addrs=127.3.89.254:44325
--builtin_ntp_servers=127.3.89.212:46353
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260809 15:39:39.523260 22394 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 15:39:39.523494 22394 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 15:39:39.523531 22394 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 15:39:39.527516 22394 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 15:39:39.527573 22394 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 15:39:39.527662 22394 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.3.89.193
I20260809 15:39:39.533434 22394 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.3.89.212:46353
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786289732823074-3431-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786289732823074-3431-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.3.89.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786289732823074-3431-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.3.89.193
--webserver_port=0
--tserver_master_addrs=127.3.89.254:44325
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.22394
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.3.89.193
--log_dir=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786289732823074-3431-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 09 Aug 2026 14:46:55 UTC on ccycloud.quasar-yswkmf.root.comops.site
build id 4576540
I20260809 15:39:39.534572 22394 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 15:39:39.536052 22394 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 15:39:39.545449 22400 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 15:39:39.545473 22403 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 15:39:39.545580 22394 server_base.cc:1034] running on GCE node
W20260809 15:39:39.545449 22401 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 15:39:39.546195 22394 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 15:39:39.546830 22394 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 15:39:39.548022 22394 hybrid_clock.cc:648] HybridClock initialized: now 1786289979548005 us; error 31 us; skew 500 ppm
I20260809 15:39:39.549908 22394 webserver.cc:460] Webserver started at http://127.3.89.193:34117/ using document root <none> and password file <none>
I20260809 15:39:39.550499 22394 fs_manager.cc:359] Metadata directory not provided
I20260809 15:39:39.550696 22394 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 15:39:39.551011 22394 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 15:39:39.553144 22394 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskm314YR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786289732823074-3431-0/minicluster-data/ts-0/data/instance:
uuid: "98843482ad594ba69390b3ce5e0e23ce"
format_stamp: "Formatted at 2026-08-09 15:39:39 on dist-test-slave-2w3w"
server_key: "dfbab39275a3a577fd7b1304ddea2f0b"
server_key_iv: "980580fef74be083838c097a33acc0f0"
server_key_version: "encryptionkey@0"
I20260809 15:39:39.553726 22394 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskm314YR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786289732823074-3431-0/minicluster-data/ts-0/wal/instance:
uuid: "98843482ad594ba69390b3ce5e0e23ce"
format_stamp: "Formatted at 2026-08-09 15:39:39 on dist-test-slave-2w3w"
server_key: "dfbab39275a3a577fd7b1304ddea2f0b"
server_key_iv: "980580fef74be083838c097a33acc0f0"
server_key_version: "encryptionkey@0"
I20260809 15:39:39.557439 22394 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.005s sys 0.000s
I20260809 15:39:39.559875 22409 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 15:39:39.561061 22394 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260809 15:39:39.561211 22394 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskm314YR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786289732823074-3431-0/minicluster-data/ts-0/data,/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786289732823074-3431-0/minicluster-data/ts-0/wal
uuid: "98843482ad594ba69390b3ce5e0e23ce"
format_stamp: "Formatted at 2026-08-09 15:39:39 on dist-test-slave-2w3w"
server_key: "dfbab39275a3a577fd7b1304ddea2f0b"
server_key_iv: "980580fef74be083838c097a33acc0f0"
server_key_version: "encryptionkey@0"
I20260809 15:39:39.561326 22394 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskm314YR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786289732823074-3431-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskm314YR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786289732823074-3431-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskm314YR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786289732823074-3431-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 15:39:39.574276 22394 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 15:39:39.574924 22394 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 15:39:39.575174 22394 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 15:39:39.575768 22394 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 15:39:39.576666 22394 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 15:39:39.576740 22394 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 15:39:39.576812 22394 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 15:39:39.576854 22394 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 15:39:39.585749 22394 rpc_server.cc:307] RPC server started. Bound to: 127.3.89.193:44355
I20260809 15:39:39.585779 22533 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.3.89.193:44355 every 8 connection(s)
I20260809 15:39:39.586617 22394 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskm314YR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786289732823074-3431-0/minicluster-data/ts-0/data/info.pb
I20260809 15:39:39.594907 3431 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskm314YR/build/debug/bin/kudu as pid 22394
I20260809 15:39:39.595072 3431 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskm314YR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786289732823074-3431-0/minicluster-data/ts-0/wal/instance
I20260809 15:39:39.595417 3431 external_mini_cluster.cc:1428] Setting key f59099b85f898f5dd751392ef7c00521
I20260809 15:39:39.596284 22369 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:39:39.588439 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.89.193:33549 (local address 127.3.89.254:44325)
0809 15:39:39.588590 (+ 151us) server_negotiation.cc:206] Beginning negotiation
0809 15:39:39.588593 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0809 15:39:39.589596 (+ 1003us) server_negotiation.cc:407] Connection header received
0809 15:39:39.590943 (+ 1347us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 15:39:39.590945 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 15:39:39.591025 (+ 80us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 15:39:39.591094 (+ 69us) server_negotiation.cc:226] Negotiated authn=SASL
0809 15:39:39.592197 (+ 1103us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:39:39.593007 (+ 810us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:39:39.594170 (+ 1163us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:39:39.594420 (+ 250us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:39:39.594528 (+ 108us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:39:39.595761 (+ 1233us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 15:39:39.595765 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 15:39:39.595767 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 15:39:39.595780 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0809 15:39:39.595809 (+ 29us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 15:39:39.595819 (+ 10us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 15:39:39.595898 (+ 79us) server_negotiation.cc:1035] Waiting for connection context
0809 15:39:39.596061 (+ 163us) server_negotiation.cc:299] Negotiation successful
0809 15:39:39.596145 (+ 84us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":63}
I20260809 15:39:39.597227 22537 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:39:39.588787 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.89.254:44325 (local address 127.3.89.193:33549)
0809 15:39:39.589448 (+ 661us) negotiation.cc:107] Waiting for socket to connect
0809 15:39:39.589489 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0809 15:39:39.590731 (+ 1242us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 15:39:39.591293 (+ 562us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 15:39:39.591299 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 15:39:39.591378 (+ 79us) client_negotiation.cc:189] Negotiated authn=SASL
0809 15:39:39.592019 (+ 641us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:39:39.592036 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:39:39.593137 (+ 1101us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:39:39.593140 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:39:39.594036 (+ 896us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:39:39.594046 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:39:39.594580 (+ 534us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:39:39.594585 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:39:39.594786 (+ 201us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:39:39.595433 (+ 647us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0809 15:39:39.595435 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0809 15:39:39.595500 (+ 65us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0809 15:39:39.595587 (+ 87us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 15:39:39.595912 (+ 325us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 15:39:39.595923 (+ 11us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 15:39:39.595929 (+ 6us) client_negotiation.cc:769] Sending connection context
0809 15:39:39.596051 (+ 122us) client_negotiation.cc:240] Negotiation successful
0809 15:39:39.596310 (+ 259us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":290,"thread_start_us":119,"threads_started":1}
I20260809 15:39:39.597469 3431 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskm314YR/build/debug/bin/kudu
/tmp/dist-test-taskm314YR/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786289732823074-3431-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786289732823074-3431-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786289732823074-3431-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786289732823074-3431-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.3.89.194:0
--local_ip_for_outbound_sockets=127.3.89.194
--webserver_interface=127.3.89.194
--webserver_port=0
--tserver_master_addrs=127.3.89.254:44325
--builtin_ntp_servers=127.3.89.212:46353
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260809 15:39:39.600099 22534 heartbeater.cc:347] Connected to a master server at 127.3.89.254:44325
I20260809 15:39:39.600360 22534 heartbeater.cc:464] Registering TS with master...
I20260809 15:39:39.600971 22534 heartbeater.cc:511] Master 127.3.89.254:44325 requested a full tablet report, sending...
I20260809 15:39:39.602557 22326 ts_manager.cc:194] Registered new tserver with Master: 98843482ad594ba69390b3ce5e0e23ce (127.3.89.193:44355)
I20260809 15:39:39.603790 22326 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.3.89.193:33549
W20260809 15:39:39.708165 22539 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 15:39:39.708446 22539 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 15:39:39.708514 22539 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 15:39:39.712509 22539 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 15:39:39.712598 22539 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 15:39:39.712699 22539 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.3.89.194
I20260809 15:39:39.717525 22539 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.3.89.212:46353
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786289732823074-3431-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786289732823074-3431-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.3.89.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786289732823074-3431-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.3.89.194
--webserver_port=0
--tserver_master_addrs=127.3.89.254:44325
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.22539
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.3.89.194
--log_dir=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786289732823074-3431-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 09 Aug 2026 14:46:55 UTC on ccycloud.quasar-yswkmf.root.comops.site
build id 4576540
I20260809 15:39:39.718603 22539 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 15:39:39.719985 22539 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 15:39:39.729234 22545 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 15:39:39.729287 22548 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 15:39:39.729233 22546 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 15:39:39.729622 22539 server_base.cc:1034] running on GCE node
I20260809 15:39:39.730012 22539 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 15:39:39.730600 22539 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 15:39:39.731756 22539 hybrid_clock.cc:648] HybridClock initialized: now 1786289979731738 us; error 30 us; skew 500 ppm
I20260809 15:39:39.733522 22539 webserver.cc:460] Webserver started at http://127.3.89.194:44515/ using document root <none> and password file <none>
I20260809 15:39:39.734102 22539 fs_manager.cc:359] Metadata directory not provided
I20260809 15:39:39.734162 22539 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 15:39:39.734392 22539 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 15:39:39.736447 22539 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskm314YR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786289732823074-3431-0/minicluster-data/ts-1/data/instance:
uuid: "0519b6b0a3ca4a138db8484f6c22828c"
format_stamp: "Formatted at 2026-08-09 15:39:39 on dist-test-slave-2w3w"
server_key: "3b5db57e4c26039aa9df3bd6c8dc229a"
server_key_iv: "ea458cef71204feae031f4cebaf896c7"
server_key_version: "encryptionkey@0"
I20260809 15:39:39.737000 22539 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskm314YR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786289732823074-3431-0/minicluster-data/ts-1/wal/instance:
uuid: "0519b6b0a3ca4a138db8484f6c22828c"
format_stamp: "Formatted at 2026-08-09 15:39:39 on dist-test-slave-2w3w"
server_key: "3b5db57e4c26039aa9df3bd6c8dc229a"
server_key_iv: "ea458cef71204feae031f4cebaf896c7"
server_key_version: "encryptionkey@0"
I20260809 15:39:39.740460 22539 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.001s sys 0.004s
I20260809 15:39:39.742693 22554 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 15:39:39.743872 22539 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260809 15:39:39.743990 22539 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskm314YR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786289732823074-3431-0/minicluster-data/ts-1/data,/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786289732823074-3431-0/minicluster-data/ts-1/wal
uuid: "0519b6b0a3ca4a138db8484f6c22828c"
format_stamp: "Formatted at 2026-08-09 15:39:39 on dist-test-slave-2w3w"
server_key: "3b5db57e4c26039aa9df3bd6c8dc229a"
server_key_iv: "ea458cef71204feae031f4cebaf896c7"
server_key_version: "encryptionkey@0"
I20260809 15:39:39.744092 22539 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskm314YR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786289732823074-3431-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskm314YR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786289732823074-3431-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskm314YR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786289732823074-3431-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 15:39:39.768733 22539 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 15:39:39.769438 22539 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 15:39:39.769644 22539 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 15:39:39.770257 22539 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 15:39:39.771281 22539 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 15:39:39.771332 22539 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 15:39:39.771399 22539 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 15:39:39.771440 22539 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 15:39:39.780426 22539 rpc_server.cc:307] RPC server started. Bound to: 127.3.89.194:42163
I20260809 15:39:39.780457 22676 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.3.89.194:42163 every 8 connection(s)
I20260809 15:39:39.781339 22539 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskm314YR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786289732823074-3431-0/minicluster-data/ts-1/data/info.pb
I20260809 15:39:39.784929 3431 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskm314YR/build/debug/bin/kudu as pid 22539
I20260809 15:39:39.785022 3431 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskm314YR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786289732823074-3431-0/minicluster-data/ts-1/wal/instance
I20260809 15:39:39.785323 3431 external_mini_cluster.cc:1428] Setting key 11779f54660c29b083f511fce2f608b0
I20260809 15:39:39.787487 3431 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskm314YR/build/debug/bin/kudu
/tmp/dist-test-taskm314YR/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786289732823074-3431-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786289732823074-3431-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786289732823074-3431-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786289732823074-3431-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.3.89.195:0
--local_ip_for_outbound_sockets=127.3.89.195
--webserver_interface=127.3.89.195
--webserver_port=0
--tserver_master_addrs=127.3.89.254:44325
--builtin_ntp_servers=127.3.89.212:46353
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260809 15:39:39.791555 22369 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:39:39.783085 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.89.194:44999 (local address 127.3.89.254:44325)
0809 15:39:39.783222 (+ 137us) server_negotiation.cc:206] Beginning negotiation
0809 15:39:39.783225 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0809 15:39:39.784266 (+ 1041us) server_negotiation.cc:407] Connection header received
0809 15:39:39.785691 (+ 1425us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 15:39:39.785695 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 15:39:39.785755 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 15:39:39.785842 (+ 87us) server_negotiation.cc:226] Negotiated authn=SASL
0809 15:39:39.786736 (+ 894us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:39:39.787550 (+ 814us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:39:39.789342 (+ 1792us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:39:39.789735 (+ 393us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:39:39.789855 (+ 120us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:39:39.790945 (+ 1090us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 15:39:39.790951 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 15:39:39.790974 (+ 23us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 15:39:39.790991 (+ 17us) server_negotiation.cc:922] calling sasl_server_start()
0809 15:39:39.791017 (+ 26us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 15:39:39.791030 (+ 13us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 15:39:39.791112 (+ 82us) server_negotiation.cc:1035] Waiting for connection context
0809 15:39:39.791260 (+ 148us) server_negotiation.cc:299] Negotiation successful
0809 15:39:39.791347 (+ 87us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
I20260809 15:39:39.792230 22679 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:39:39.783492 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.89.254:44325 (local address 127.3.89.194:44999)
0809 15:39:39.784125 (+ 633us) negotiation.cc:107] Waiting for socket to connect
0809 15:39:39.784164 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0809 15:39:39.785492 (+ 1328us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 15:39:39.786043 (+ 551us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 15:39:39.786049 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 15:39:39.786127 (+ 78us) client_negotiation.cc:189] Negotiated authn=SASL
0809 15:39:39.786575 (+ 448us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:39:39.786616 (+ 41us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:39:39.787719 (+ 1103us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:39:39.787725 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:39:39.789181 (+ 1456us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:39:39.789196 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:39:39.789892 (+ 696us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:39:39.789895 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:39:39.790040 (+ 145us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:39:39.790686 (+ 646us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0809 15:39:39.790687 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0809 15:39:39.790724 (+ 37us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0809 15:39:39.790783 (+ 59us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 15:39:39.791115 (+ 332us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 15:39:39.791122 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 15:39:39.791127 (+ 5us) client_negotiation.cc:769] Sending connection context
0809 15:39:39.791223 (+ 96us) client_negotiation.cc:240] Negotiation successful
0809 15:39:39.791428 (+ 205us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":254,"thread_start_us":120,"threads_started":1}
I20260809 15:39:39.793381 22677 heartbeater.cc:347] Connected to a master server at 127.3.89.254:44325
I20260809 15:39:39.793619 22677 heartbeater.cc:464] Registering TS with master...
I20260809 15:39:39.794107 22677 heartbeater.cc:511] Master 127.3.89.254:44325 requested a full tablet report, sending...
I20260809 15:39:39.795171 22326 ts_manager.cc:194] Registered new tserver with Master: 0519b6b0a3ca4a138db8484f6c22828c (127.3.89.194:42163)
I20260809 15:39:39.796082 22326 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.3.89.194:44999
W20260809 15:39:39.897708 22680 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 15:39:39.897938 22680 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 15:39:39.897971 22680 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 15:39:39.902046 22680 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 15:39:39.902096 22680 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 15:39:39.902148 22680 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.3.89.195
I20260809 15:39:39.906826 22680 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.3.89.212:46353
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786289732823074-3431-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786289732823074-3431-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.3.89.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786289732823074-3431-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.3.89.195
--webserver_port=0
--tserver_master_addrs=127.3.89.254:44325
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.22680
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.3.89.195
--log_dir=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786289732823074-3431-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 09 Aug 2026 14:46:55 UTC on ccycloud.quasar-yswkmf.root.comops.site
build id 4576540
I20260809 15:39:39.907829 22680 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 15:39:39.909075 22680 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 15:39:39.918426 22689 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 15:39:39.918445 22688 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 15:39:39.918542 22691 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 15:39:39.918668 22680 server_base.cc:1034] running on GCE node
I20260809 15:39:39.919114 22680 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 15:39:39.919734 22680 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 15:39:39.920897 22680 hybrid_clock.cc:648] HybridClock initialized: now 1786289979920881 us; error 28 us; skew 500 ppm
I20260809 15:39:39.922724 22680 webserver.cc:460] Webserver started at http://127.3.89.195:43933/ using document root <none> and password file <none>
I20260809 15:39:39.923328 22680 fs_manager.cc:359] Metadata directory not provided
I20260809 15:39:39.923398 22680 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 15:39:39.923628 22680 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 15:39:39.925572 22680 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskm314YR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786289732823074-3431-0/minicluster-data/ts-2/data/instance:
uuid: "eac1fb0c078b4f2980b8ad8ff9725d71"
format_stamp: "Formatted at 2026-08-09 15:39:39 on dist-test-slave-2w3w"
server_key: "1571ecaf83328e6f77818585f75ba09e"
server_key_iv: "98e08244cbdc8849319c1d98a8b14c32"
server_key_version: "encryptionkey@0"
I20260809 15:39:39.926131 22680 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskm314YR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786289732823074-3431-0/minicluster-data/ts-2/wal/instance:
uuid: "eac1fb0c078b4f2980b8ad8ff9725d71"
format_stamp: "Formatted at 2026-08-09 15:39:39 on dist-test-slave-2w3w"
server_key: "1571ecaf83328e6f77818585f75ba09e"
server_key_iv: "98e08244cbdc8849319c1d98a8b14c32"
server_key_version: "encryptionkey@0"
I20260809 15:39:39.929721 22680 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.006s sys 0.000s
I20260809 15:39:39.932004 22697 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 15:39:39.933136 22680 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260809 15:39:39.933276 22680 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskm314YR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786289732823074-3431-0/minicluster-data/ts-2/data,/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786289732823074-3431-0/minicluster-data/ts-2/wal
uuid: "eac1fb0c078b4f2980b8ad8ff9725d71"
format_stamp: "Formatted at 2026-08-09 15:39:39 on dist-test-slave-2w3w"
server_key: "1571ecaf83328e6f77818585f75ba09e"
server_key_iv: "98e08244cbdc8849319c1d98a8b14c32"
server_key_version: "encryptionkey@0"
I20260809 15:39:39.933403 22680 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskm314YR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786289732823074-3431-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskm314YR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786289732823074-3431-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskm314YR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786289732823074-3431-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 15:39:39.958600 22680 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 15:39:39.959388 22680 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 15:39:39.959587 22680 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 15:39:39.960182 22680 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 15:39:39.961104 22680 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 15:39:39.961175 22680 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 15:39:39.961247 22680 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 15:39:39.961300 22680 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 15:39:39.970113 22680 rpc_server.cc:307] RPC server started. Bound to: 127.3.89.195:42417
I20260809 15:39:39.970140 22822 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.3.89.195:42417 every 8 connection(s)
I20260809 15:39:39.971046 22680 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskm314YR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786289732823074-3431-0/minicluster-data/ts-2/data/info.pb
I20260809 15:39:39.975065 3431 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskm314YR/build/debug/bin/kudu as pid 22680
I20260809 15:39:39.975201 3431 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskm314YR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786289732823074-3431-0/minicluster-data/ts-2/wal/instance
I20260809 15:39:39.975533 3431 external_mini_cluster.cc:1428] Setting key 3f5bc685a918a4455dabafafdd718ab4
I20260809 15:39:39.980821 22369 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:39:39.972843 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.89.195:35093 (local address 127.3.89.254:44325)
0809 15:39:39.972990 (+ 147us) server_negotiation.cc:206] Beginning negotiation
0809 15:39:39.972994 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 15:39:39.974029 (+ 1035us) server_negotiation.cc:407] Connection header received
0809 15:39:39.975748 (+ 1719us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 15:39:39.975751 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 15:39:39.975800 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 15:39:39.975915 (+ 115us) server_negotiation.cc:226] Negotiated authn=SASL
0809 15:39:39.977116 (+ 1201us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:39:39.977757 (+ 641us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:39:39.978898 (+ 1141us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:39:39.979185 (+ 287us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:39:39.979306 (+ 121us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:39:39.980342 (+ 1036us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 15:39:39.980346 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 15:39:39.980348 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 15:39:39.980365 (+ 17us) server_negotiation.cc:922] calling sasl_server_start()
0809 15:39:39.980384 (+ 19us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 15:39:39.980392 (+ 8us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 15:39:39.980464 (+ 72us) server_negotiation.cc:1035] Waiting for connection context
0809 15:39:39.980624 (+ 160us) server_negotiation.cc:299] Negotiation successful
0809 15:39:39.980682 (+ 58us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":56}
I20260809 15:39:39.981657 22825 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:39:39.973201 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.89.254:44325 (local address 127.3.89.195:35093)
0809 15:39:39.973875 (+ 674us) negotiation.cc:107] Waiting for socket to connect
0809 15:39:39.973916 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0809 15:39:39.975504 (+ 1588us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 15:39:39.976109 (+ 605us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 15:39:39.976118 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 15:39:39.976222 (+ 104us) client_negotiation.cc:189] Negotiated authn=SASL
0809 15:39:39.976971 (+ 749us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:39:39.976996 (+ 25us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:39:39.977881 (+ 885us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:39:39.977884 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:39:39.978762 (+ 878us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:39:39.978771 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:39:39.979343 (+ 572us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:39:39.979347 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:39:39.979524 (+ 177us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:39:39.980099 (+ 575us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0809 15:39:39.980101 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0809 15:39:39.980138 (+ 37us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0809 15:39:39.980194 (+ 56us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 15:39:39.980478 (+ 284us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 15:39:39.980487 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 15:39:39.980492 (+ 5us) client_negotiation.cc:769] Sending connection context
0809 15:39:39.980623 (+ 131us) client_negotiation.cc:240] Negotiation successful
0809 15:39:39.980935 (+ 312us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":267,"thread_start_us":112,"threads_started":1}
I20260809 15:39:39.982652 22823 heartbeater.cc:347] Connected to a master server at 127.3.89.254:44325
I20260809 15:39:39.982892 22823 heartbeater.cc:464] Registering TS with master...
I20260809 15:39:39.983390 22823 heartbeater.cc:511] Master 127.3.89.254:44325 requested a full tablet report, sending...
I20260809 15:39:39.984380 22326 ts_manager.cc:194] Registered new tserver with Master: eac1fb0c078b4f2980b8ad8ff9725d71 (127.3.89.195:42417)
I20260809 15:39:39.984946 22326 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.3.89.195:35093
I20260809 15:39:39.989197 3431 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260809 15:39:39.994602 22369 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:39:39.991194 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53840 (local address 127.3.89.254:44325)
0809 15:39:39.991320 (+ 126us) server_negotiation.cc:206] Beginning negotiation
0809 15:39:39.991324 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 15:39:39.991510 (+ 186us) server_negotiation.cc:407] Connection header received
0809 15:39:39.991599 (+ 89us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 15:39:39.991601 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 15:39:39.991636 (+ 35us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 15:39:39.991696 (+ 60us) server_negotiation.cc:226] Negotiated authn=SASL
0809 15:39:39.991904 (+ 208us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:39:39.992400 (+ 496us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:39:39.993199 (+ 799us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:39:39.993417 (+ 218us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:39:39.993496 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:39:39.994268 (+ 772us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 15:39:39.994271 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 15:39:39.994273 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 15:39:39.994283 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0809 15:39:39.994301 (+ 18us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 15:39:39.994308 (+ 7us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 15:39:39.994365 (+ 57us) server_negotiation.cc:1035] Waiting for connection context
0809 15:39:39.994460 (+ 95us) server_negotiation.cc:299] Negotiation successful
0809 15:39:39.994505 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":56}
I20260809 15:39:39.997364 22326 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:53840:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260809 15:39:40.000578 22326 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260809 15:39:40.015185 22835 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:39:40.009013 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.89.195:42417 (local address 127.0.0.1:51750)
0809 15:39:40.009754 (+ 741us) negotiation.cc:107] Waiting for socket to connect
0809 15:39:40.009805 (+ 51us) client_negotiation.cc:174] Beginning negotiation
0809 15:39:40.009946 (+ 141us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 15:39:40.010771 (+ 825us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 15:39:40.010780 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 15:39:40.010806 (+ 26us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 15:39:40.010941 (+ 135us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:39:40.010947 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:39:40.012421 (+ 1474us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:39:40.012424 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:39:40.013718 (+ 1294us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:39:40.013729 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:39:40.014596 (+ 867us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:39:40.014600 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:39:40.014704 (+ 104us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:39:40.014757 (+ 53us) client_negotiation.cc:769] Sending connection context
0809 15:39:40.014879 (+ 122us) client_negotiation.cc:240] Negotiation successful
0809 15:39:40.015017 (+ 138us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":565,"mutex_wait_us":82,"thread_start_us":120,"threads_started":1}
I20260809 15:39:40.015415 22839 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:39:40.009963 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58166 (local address 127.3.89.193:44355)
0809 15:39:40.010587 (+ 624us) server_negotiation.cc:206] Beginning negotiation
0809 15:39:40.010599 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0809 15:39:40.010621 (+ 22us) server_negotiation.cc:407] Connection header received
0809 15:39:40.010693 (+ 72us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 15:39:40.010696 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 15:39:40.010818 (+ 122us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 15:39:40.010967 (+ 149us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 15:39:40.011214 (+ 247us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:39:40.012079 (+ 865us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:39:40.013872 (+ 1793us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:39:40.014635 (+ 763us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:39:40.014760 (+ 125us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:39:40.014952 (+ 192us) server_negotiation.cc:1035] Waiting for connection context
0809 15:39:40.015140 (+ 188us) server_negotiation.cc:299] Negotiation successful
0809 15:39:40.015238 (+ 98us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":516,"thread_start_us":124,"threads_started":1}
I20260809 15:39:40.015589 22837 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:39:40.009292 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.89.193:44355 (local address 127.0.0.1:58166)
0809 15:39:40.009929 (+ 637us) negotiation.cc:107] Waiting for socket to connect
0809 15:39:40.009943 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0809 15:39:40.010032 (+ 89us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 15:39:40.010947 (+ 915us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 15:39:40.010950 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 15:39:40.010988 (+ 38us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 15:39:40.011097 (+ 109us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:39:40.011104 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:39:40.012189 (+ 1085us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:39:40.012194 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:39:40.013734 (+ 1540us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:39:40.013740 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:39:40.014881 (+ 1141us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:39:40.014886 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:39:40.014995 (+ 109us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:39:40.015015 (+ 20us) client_negotiation.cc:769] Sending connection context
0809 15:39:40.015092 (+ 77us) client_negotiation.cc:240] Negotiation successful
0809 15:39:40.015147 (+ 55us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":561,"mutex_wait_us":104,"thread_start_us":104,"threads_started":1}
I20260809 15:39:40.015709 22836 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:39:40.009224 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51750 (local address 127.3.89.195:42417)
0809 15:39:40.009761 (+ 537us) server_negotiation.cc:206] Beginning negotiation
0809 15:39:40.009781 (+ 20us) server_negotiation.cc:399] Waiting for connection header
0809 15:39:40.010120 (+ 339us) server_negotiation.cc:407] Connection header received
0809 15:39:40.010205 (+ 85us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 15:39:40.010210 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 15:39:40.010419 (+ 209us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 15:39:40.010580 (+ 161us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 15:39:40.011117 (+ 537us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:39:40.012303 (+ 1186us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:39:40.013854 (+ 1551us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:39:40.014464 (+ 610us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:39:40.014594 (+ 130us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:39:40.014803 (+ 209us) server_negotiation.cc:1035] Waiting for connection context
0809 15:39:40.015372 (+ 569us) server_negotiation.cc:299] Negotiation successful
0809 15:39:40.015543 (+ 171us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":430,"thread_start_us":166,"threads_started":1}
I20260809 15:39:40.015897 22838 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:39:40.009792 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.89.194:42163 (local address 127.0.0.1:48942)
0809 15:39:40.010642 (+ 850us) negotiation.cc:107] Waiting for socket to connect
0809 15:39:40.010655 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0809 15:39:40.010741 (+ 86us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 15:39:40.011436 (+ 695us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 15:39:40.011439 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 15:39:40.011457 (+ 18us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 15:39:40.011543 (+ 86us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:39:40.011548 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:39:40.012788 (+ 1240us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:39:40.012791 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:39:40.013902 (+ 1111us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:39:40.013913 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:39:40.015153 (+ 1240us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:39:40.015157 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:39:40.015243 (+ 86us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:39:40.015264 (+ 21us) client_negotiation.cc:769] Sending connection context
0809 15:39:40.015660 (+ 396us) client_negotiation.cc:240] Negotiation successful
0809 15:39:40.015754 (+ 94us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":786,"mutex_wait_us":68,"thread_start_us":103,"threads_started":1}
I20260809 15:39:40.016315 22841 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:39:40.010221 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48942 (local address 127.3.89.194:42163)
0809 15:39:40.011002 (+ 781us) server_negotiation.cc:206] Beginning negotiation
0809 15:39:40.011019 (+ 17us) server_negotiation.cc:399] Waiting for connection header
0809 15:39:40.011044 (+ 25us) server_negotiation.cc:407] Connection header received
0809 15:39:40.011120 (+ 76us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 15:39:40.011124 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 15:39:40.011294 (+ 170us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 15:39:40.011442 (+ 148us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 15:39:40.011638 (+ 196us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:39:40.012634 (+ 996us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:39:40.014024 (+ 1390us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:39:40.014792 (+ 768us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:39:40.014888 (+ 96us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:39:40.015086 (+ 198us) server_negotiation.cc:1035] Waiting for connection context
0809 15:39:40.016114 (+ 1028us) server_negotiation.cc:299] Negotiation successful
0809 15:39:40.016199 (+ 85us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":657,"thread_start_us":141,"threads_started":1}
I20260809 15:39:40.018294 22461 tablet_service.cc:1467] Processing CreateTablet for tablet 86c8bccdc6c1402fa833c810796b0276 (DEFAULT_TABLE table=test-table [id=71122d9aa0b7424386e707986aba1af9]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 15:39:40.018292 22747 tablet_service.cc:1467] Processing CreateTablet for tablet 86c8bccdc6c1402fa833c810796b0276 (DEFAULT_TABLE table=test-table [id=71122d9aa0b7424386e707986aba1af9]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 15:39:40.019276 22747 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 86c8bccdc6c1402fa833c810796b0276. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 15:39:40.019301 22461 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 86c8bccdc6c1402fa833c810796b0276. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 15:39:40.021564 22603 tablet_service.cc:1467] Processing CreateTablet for tablet 86c8bccdc6c1402fa833c810796b0276 (DEFAULT_TABLE table=test-table [id=71122d9aa0b7424386e707986aba1af9]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 15:39:40.022554 22603 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 86c8bccdc6c1402fa833c810796b0276. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 15:39:40.025143 22842 tablet_bootstrap.cc:492] T 86c8bccdc6c1402fa833c810796b0276 P 98843482ad594ba69390b3ce5e0e23ce: Bootstrap starting.
I20260809 15:39:40.026588 22843 tablet_bootstrap.cc:492] T 86c8bccdc6c1402fa833c810796b0276 P eac1fb0c078b4f2980b8ad8ff9725d71: Bootstrap starting.
I20260809 15:39:40.027488 22842 tablet_bootstrap.cc:654] T 86c8bccdc6c1402fa833c810796b0276 P 98843482ad594ba69390b3ce5e0e23ce: Neither blocks nor log segments found. Creating new log.
I20260809 15:39:40.028335 22842 log.cc:824] T 86c8bccdc6c1402fa833c810796b0276 P 98843482ad594ba69390b3ce5e0e23ce: Log is configured to *not* fsync() on all Append() calls
I20260809 15:39:40.028683 22843 tablet_bootstrap.cc:654] T 86c8bccdc6c1402fa833c810796b0276 P eac1fb0c078b4f2980b8ad8ff9725d71: Neither blocks nor log segments found. Creating new log.
I20260809 15:39:40.029106 22844 tablet_bootstrap.cc:492] T 86c8bccdc6c1402fa833c810796b0276 P 0519b6b0a3ca4a138db8484f6c22828c: Bootstrap starting.
I20260809 15:39:40.029524 22843 log.cc:824] T 86c8bccdc6c1402fa833c810796b0276 P eac1fb0c078b4f2980b8ad8ff9725d71: Log is configured to *not* fsync() on all Append() calls
I20260809 15:39:40.030432 22842 tablet_bootstrap.cc:492] T 86c8bccdc6c1402fa833c810796b0276 P 98843482ad594ba69390b3ce5e0e23ce: No bootstrap required, opened a new log
I20260809 15:39:40.030617 22842 ts_tablet_manager.cc:1397] T 86c8bccdc6c1402fa833c810796b0276 P 98843482ad594ba69390b3ce5e0e23ce: Time spent bootstrapping tablet: real 0.006s user 0.004s sys 0.000s
I20260809 15:39:40.031226 22844 tablet_bootstrap.cc:654] T 86c8bccdc6c1402fa833c810796b0276 P 0519b6b0a3ca4a138db8484f6c22828c: Neither blocks nor log segments found. Creating new log.
I20260809 15:39:40.031525 22843 tablet_bootstrap.cc:492] T 86c8bccdc6c1402fa833c810796b0276 P eac1fb0c078b4f2980b8ad8ff9725d71: No bootstrap required, opened a new log
I20260809 15:39:40.031678 22843 ts_tablet_manager.cc:1397] T 86c8bccdc6c1402fa833c810796b0276 P eac1fb0c078b4f2980b8ad8ff9725d71: Time spent bootstrapping tablet: real 0.005s user 0.004s sys 0.000s
I20260809 15:39:40.032016 22844 log.cc:824] T 86c8bccdc6c1402fa833c810796b0276 P 0519b6b0a3ca4a138db8484f6c22828c: Log is configured to *not* fsync() on all Append() calls
I20260809 15:39:40.033591 22844 tablet_bootstrap.cc:492] T 86c8bccdc6c1402fa833c810796b0276 P 0519b6b0a3ca4a138db8484f6c22828c: No bootstrap required, opened a new log
I20260809 15:39:40.033780 22844 ts_tablet_manager.cc:1397] T 86c8bccdc6c1402fa833c810796b0276 P 0519b6b0a3ca4a138db8484f6c22828c: Time spent bootstrapping tablet: real 0.005s user 0.003s sys 0.000s
I20260809 15:39:40.034997 22842 raft_consensus.cc:348] T 86c8bccdc6c1402fa833c810796b0276 P 98843482ad594ba69390b3ce5e0e23ce [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eac1fb0c078b4f2980b8ad8ff9725d71" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 42417 } } peers { permanent_uuid: "98843482ad594ba69390b3ce5e0e23ce" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 44355 } } peers { permanent_uuid: "0519b6b0a3ca4a138db8484f6c22828c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42163 } }
I20260809 15:39:40.035269 22842 raft_consensus.cc:374] T 86c8bccdc6c1402fa833c810796b0276 P 98843482ad594ba69390b3ce5e0e23ce [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 15:39:40.035326 22842 raft_consensus.cc:729] T 86c8bccdc6c1402fa833c810796b0276 P 98843482ad594ba69390b3ce5e0e23ce [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 98843482ad594ba69390b3ce5e0e23ce, State: Initialized, Role: FOLLOWER
I20260809 15:39:40.035753 22842 consensus_queue.cc:260] T 86c8bccdc6c1402fa833c810796b0276 P 98843482ad594ba69390b3ce5e0e23ce [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eac1fb0c078b4f2980b8ad8ff9725d71" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 42417 } } peers { permanent_uuid: "98843482ad594ba69390b3ce5e0e23ce" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 44355 } } peers { permanent_uuid: "0519b6b0a3ca4a138db8484f6c22828c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42163 } }
I20260809 15:39:40.035699 22843 raft_consensus.cc:348] T 86c8bccdc6c1402fa833c810796b0276 P eac1fb0c078b4f2980b8ad8ff9725d71 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eac1fb0c078b4f2980b8ad8ff9725d71" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 42417 } } peers { permanent_uuid: "98843482ad594ba69390b3ce5e0e23ce" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 44355 } } peers { permanent_uuid: "0519b6b0a3ca4a138db8484f6c22828c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42163 } }
I20260809 15:39:40.035919 22843 raft_consensus.cc:374] T 86c8bccdc6c1402fa833c810796b0276 P eac1fb0c078b4f2980b8ad8ff9725d71 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 15:39:40.035974 22843 raft_consensus.cc:729] T 86c8bccdc6c1402fa833c810796b0276 P eac1fb0c078b4f2980b8ad8ff9725d71 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: eac1fb0c078b4f2980b8ad8ff9725d71, State: Initialized, Role: FOLLOWER
I20260809 15:39:40.036445 22843 consensus_queue.cc:260] T 86c8bccdc6c1402fa833c810796b0276 P eac1fb0c078b4f2980b8ad8ff9725d71 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eac1fb0c078b4f2980b8ad8ff9725d71" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 42417 } } peers { permanent_uuid: "98843482ad594ba69390b3ce5e0e23ce" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 44355 } } peers { permanent_uuid: "0519b6b0a3ca4a138db8484f6c22828c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42163 } }
I20260809 15:39:40.036773 22842 ts_tablet_manager.cc:1428] T 86c8bccdc6c1402fa833c810796b0276 P 98843482ad594ba69390b3ce5e0e23ce: Time spent starting tablet: real 0.006s user 0.003s sys 0.004s
I20260809 15:39:40.037113 22534 heartbeater.cc:503] Master 127.3.89.254:44325 was elected leader, sending a full tablet report...
I20260809 15:39:40.037381 22823 heartbeater.cc:503] Master 127.3.89.254:44325 was elected leader, sending a full tablet report...
I20260809 15:39:40.037423 22844 raft_consensus.cc:348] T 86c8bccdc6c1402fa833c810796b0276 P 0519b6b0a3ca4a138db8484f6c22828c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eac1fb0c078b4f2980b8ad8ff9725d71" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 42417 } } peers { permanent_uuid: "98843482ad594ba69390b3ce5e0e23ce" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 44355 } } peers { permanent_uuid: "0519b6b0a3ca4a138db8484f6c22828c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42163 } }
I20260809 15:39:40.037669 22844 raft_consensus.cc:374] T 86c8bccdc6c1402fa833c810796b0276 P 0519b6b0a3ca4a138db8484f6c22828c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 15:39:40.037732 22844 raft_consensus.cc:729] T 86c8bccdc6c1402fa833c810796b0276 P 0519b6b0a3ca4a138db8484f6c22828c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0519b6b0a3ca4a138db8484f6c22828c, State: Initialized, Role: FOLLOWER
I20260809 15:39:40.038153 22844 consensus_queue.cc:260] T 86c8bccdc6c1402fa833c810796b0276 P 0519b6b0a3ca4a138db8484f6c22828c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eac1fb0c078b4f2980b8ad8ff9725d71" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 42417 } } peers { permanent_uuid: "98843482ad594ba69390b3ce5e0e23ce" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 44355 } } peers { permanent_uuid: "0519b6b0a3ca4a138db8484f6c22828c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42163 } }
I20260809 15:39:40.038534 22843 ts_tablet_manager.cc:1428] T 86c8bccdc6c1402fa833c810796b0276 P eac1fb0c078b4f2980b8ad8ff9725d71: Time spent starting tablet: real 0.007s user 0.003s sys 0.003s
I20260809 15:39:40.039067 22844 ts_tablet_manager.cc:1428] T 86c8bccdc6c1402fa833c810796b0276 P 0519b6b0a3ca4a138db8484f6c22828c: Time spent starting tablet: real 0.005s user 0.002s sys 0.005s
I20260809 15:39:40.039147 22677 heartbeater.cc:503] Master 127.3.89.254:44325 was elected leader, sending a full tablet report...
W20260809 15:39:40.087729 22535 tablet.cc:2369] T 86c8bccdc6c1402fa833c810796b0276 P 98843482ad594ba69390b3ce5e0e23ce: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260809 15:39:40.222039 22824 tablet.cc:2369] T 86c8bccdc6c1402fa833c810796b0276 P eac1fb0c078b4f2980b8ad8ff9725d71: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260809 15:39:40.225212 22849 raft_consensus.cc:482] T 86c8bccdc6c1402fa833c810796b0276 P eac1fb0c078b4f2980b8ad8ff9725d71 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 15:39:40.225386 22849 raft_consensus.cc:504] T 86c8bccdc6c1402fa833c810796b0276 P eac1fb0c078b4f2980b8ad8ff9725d71 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eac1fb0c078b4f2980b8ad8ff9725d71" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 42417 } } peers { permanent_uuid: "98843482ad594ba69390b3ce5e0e23ce" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 44355 } } peers { permanent_uuid: "0519b6b0a3ca4a138db8484f6c22828c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42163 } }
I20260809 15:39:40.226503 22849 leader_election.cc:288] T 86c8bccdc6c1402fa833c810796b0276 P eac1fb0c078b4f2980b8ad8ff9725d71 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 98843482ad594ba69390b3ce5e0e23ce (127.3.89.193:44355), 0519b6b0a3ca4a138db8484f6c22828c (127.3.89.194:42163)
I20260809 15:39:40.230154 22825 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:39:40.226751 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.89.194:42163 (local address 127.3.89.195:53605)
0809 15:39:40.226985 (+ 234us) negotiation.cc:107] Waiting for socket to connect
0809 15:39:40.227004 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0809 15:39:40.227103 (+ 99us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 15:39:40.227422 (+ 319us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 15:39:40.227424 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 15:39:40.227443 (+ 19us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 15:39:40.227568 (+ 125us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:39:40.227574 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:39:40.228311 (+ 737us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:39:40.228317 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:39:40.229100 (+ 783us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:39:40.229109 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:39:40.229728 (+ 619us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:39:40.229732 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:39:40.229834 (+ 102us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:39:40.229863 (+ 29us) client_negotiation.cc:769] Sending connection context
0809 15:39:40.229951 (+ 88us) client_negotiation.cc:240] Negotiation successful
0809 15:39:40.230023 (+ 72us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":131,"mutex_wait_us":1}
I20260809 15:39:40.230152 22841 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:39:40.226888 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.89.195:53605 (local address 127.3.89.194:42163)
0809 15:39:40.227042 (+ 154us) server_negotiation.cc:206] Beginning negotiation
0809 15:39:40.227047 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 15:39:40.227065 (+ 18us) server_negotiation.cc:407] Connection header received
0809 15:39:40.227260 (+ 195us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 15:39:40.227263 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 15:39:40.227313 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 15:39:40.227399 (+ 86us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 15:39:40.227687 (+ 288us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:39:40.228193 (+ 506us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:39:40.229227 (+ 1034us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:39:40.229608 (+ 381us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:39:40.229746 (+ 138us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:39:40.229800 (+ 54us) server_negotiation.cc:1035] Waiting for connection context
0809 15:39:40.229968 (+ 168us) server_negotiation.cc:299] Negotiation successful
0809 15:39:40.230033 (+ 65us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":54}
I20260809 15:39:40.230405 22839 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:39:40.226915 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.89.195:43475 (local address 127.3.89.193:44355)
0809 15:39:40.227067 (+ 152us) server_negotiation.cc:206] Beginning negotiation
0809 15:39:40.227072 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 15:39:40.227358 (+ 286us) server_negotiation.cc:407] Connection header received
0809 15:39:40.227513 (+ 155us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 15:39:40.227521 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 15:39:40.227566 (+ 45us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 15:39:40.227646 (+ 80us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 15:39:40.227963 (+ 317us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:39:40.228455 (+ 492us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:39:40.229462 (+ 1007us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:39:40.229834 (+ 372us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:39:40.229963 (+ 129us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:39:40.230006 (+ 43us) server_negotiation.cc:1035] Waiting for connection context
0809 15:39:40.230183 (+ 177us) server_negotiation.cc:299] Negotiation successful
0809 15:39:40.230273 (+ 90us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":70}
I20260809 15:39:40.230573 22851 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:39:40.226752 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.89.193:44355 (local address 127.3.89.195:43475)
0809 15:39:40.227243 (+ 491us) negotiation.cc:107] Waiting for socket to connect
0809 15:39:40.227254 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0809 15:39:40.227385 (+ 131us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 15:39:40.227666 (+ 281us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 15:39:40.227669 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 15:39:40.227684 (+ 15us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 15:39:40.227840 (+ 156us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:39:40.227845 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:39:40.228563 (+ 718us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:39:40.228568 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:39:40.229333 (+ 765us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:39:40.229341 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:39:40.229954 (+ 613us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:39:40.229957 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:39:40.230054 (+ 97us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:39:40.230077 (+ 23us) client_negotiation.cc:769] Sending connection context
0809 15:39:40.230165 (+ 88us) client_negotiation.cc:240] Negotiation successful
0809 15:39:40.230362 (+ 197us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":439,"mutex_wait_us":37,"thread_start_us":160,"threads_started":1}
I20260809 15:39:40.230851 22625 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "86c8bccdc6c1402fa833c810796b0276" candidate_uuid: "eac1fb0c078b4f2980b8ad8ff9725d71" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0519b6b0a3ca4a138db8484f6c22828c" is_pre_election: true
I20260809 15:39:40.231190 22625 raft_consensus.cc:2454] T 86c8bccdc6c1402fa833c810796b0276 P 0519b6b0a3ca4a138db8484f6c22828c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate eac1fb0c078b4f2980b8ad8ff9725d71 in term 0.
I20260809 15:39:40.231257 22482 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "86c8bccdc6c1402fa833c810796b0276" candidate_uuid: "eac1fb0c078b4f2980b8ad8ff9725d71" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "98843482ad594ba69390b3ce5e0e23ce" is_pre_election: true
I20260809 15:39:40.231531 22482 raft_consensus.cc:2454] T 86c8bccdc6c1402fa833c810796b0276 P 98843482ad594ba69390b3ce5e0e23ce [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate eac1fb0c078b4f2980b8ad8ff9725d71 in term 0.
I20260809 15:39:40.231783 22698 leader_election.cc:302] T 86c8bccdc6c1402fa833c810796b0276 P eac1fb0c078b4f2980b8ad8ff9725d71 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0519b6b0a3ca4a138db8484f6c22828c, eac1fb0c078b4f2980b8ad8ff9725d71; no voters:
I20260809 15:39:40.232077 22849 raft_consensus.cc:2781] T 86c8bccdc6c1402fa833c810796b0276 P eac1fb0c078b4f2980b8ad8ff9725d71 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260809 15:39:40.232152 22849 raft_consensus.cc:482] T 86c8bccdc6c1402fa833c810796b0276 P eac1fb0c078b4f2980b8ad8ff9725d71 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 15:39:40.232182 22849 raft_consensus.cc:3037] T 86c8bccdc6c1402fa833c810796b0276 P eac1fb0c078b4f2980b8ad8ff9725d71 [term 0 FOLLOWER]: Advancing to term 1
I20260809 15:39:40.233263 22849 raft_consensus.cc:504] T 86c8bccdc6c1402fa833c810796b0276 P eac1fb0c078b4f2980b8ad8ff9725d71 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eac1fb0c078b4f2980b8ad8ff9725d71" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 42417 } } peers { permanent_uuid: "98843482ad594ba69390b3ce5e0e23ce" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 44355 } } peers { permanent_uuid: "0519b6b0a3ca4a138db8484f6c22828c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42163 } }
I20260809 15:39:40.233640 22849 leader_election.cc:288] T 86c8bccdc6c1402fa833c810796b0276 P eac1fb0c078b4f2980b8ad8ff9725d71 [CANDIDATE]: Term 1 election: Requested vote from peers 98843482ad594ba69390b3ce5e0e23ce (127.3.89.193:44355), 0519b6b0a3ca4a138db8484f6c22828c (127.3.89.194:42163)
I20260809 15:39:40.234020 22482 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "86c8bccdc6c1402fa833c810796b0276" candidate_uuid: "eac1fb0c078b4f2980b8ad8ff9725d71" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "98843482ad594ba69390b3ce5e0e23ce"
I20260809 15:39:40.234133 22625 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "86c8bccdc6c1402fa833c810796b0276" candidate_uuid: "eac1fb0c078b4f2980b8ad8ff9725d71" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0519b6b0a3ca4a138db8484f6c22828c"
I20260809 15:39:40.234143 22482 raft_consensus.cc:3037] T 86c8bccdc6c1402fa833c810796b0276 P 98843482ad594ba69390b3ce5e0e23ce [term 0 FOLLOWER]: Advancing to term 1
I20260809 15:39:40.234313 22625 raft_consensus.cc:3037] T 86c8bccdc6c1402fa833c810796b0276 P 0519b6b0a3ca4a138db8484f6c22828c [term 0 FOLLOWER]: Advancing to term 1
I20260809 15:39:40.235354 22482 raft_consensus.cc:2454] T 86c8bccdc6c1402fa833c810796b0276 P 98843482ad594ba69390b3ce5e0e23ce [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate eac1fb0c078b4f2980b8ad8ff9725d71 in term 1.
I20260809 15:39:40.235558 22625 raft_consensus.cc:2454] T 86c8bccdc6c1402fa833c810796b0276 P 0519b6b0a3ca4a138db8484f6c22828c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate eac1fb0c078b4f2980b8ad8ff9725d71 in term 1.
I20260809 15:39:40.235693 22700 leader_election.cc:302] T 86c8bccdc6c1402fa833c810796b0276 P eac1fb0c078b4f2980b8ad8ff9725d71 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 98843482ad594ba69390b3ce5e0e23ce, eac1fb0c078b4f2980b8ad8ff9725d71; no voters:
I20260809 15:39:40.235868 22849 raft_consensus.cc:2781] T 86c8bccdc6c1402fa833c810796b0276 P eac1fb0c078b4f2980b8ad8ff9725d71 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 15:39:40.236104 22849 raft_consensus.cc:686] T 86c8bccdc6c1402fa833c810796b0276 P eac1fb0c078b4f2980b8ad8ff9725d71 [term 1 LEADER]: Becoming Leader. State: Replica: eac1fb0c078b4f2980b8ad8ff9725d71, State: Running, Role: LEADER
I20260809 15:39:40.236442 22849 consensus_queue.cc:237] T 86c8bccdc6c1402fa833c810796b0276 P eac1fb0c078b4f2980b8ad8ff9725d71 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eac1fb0c078b4f2980b8ad8ff9725d71" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 42417 } } peers { permanent_uuid: "98843482ad594ba69390b3ce5e0e23ce" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 44355 } } peers { permanent_uuid: "0519b6b0a3ca4a138db8484f6c22828c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42163 } }
I20260809 15:39:40.239665 22326 catalog_manager.cc:5495] T 86c8bccdc6c1402fa833c810796b0276 P eac1fb0c078b4f2980b8ad8ff9725d71 reported cstate change: term changed from 0 to 1, leader changed from <none> to eac1fb0c078b4f2980b8ad8ff9725d71 (127.3.89.195). New cstate: current_term: 1 leader_uuid: "eac1fb0c078b4f2980b8ad8ff9725d71" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eac1fb0c078b4f2980b8ad8ff9725d71" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 42417 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "98843482ad594ba69390b3ce5e0e23ce" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 44355 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "0519b6b0a3ca4a138db8484f6c22828c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42163 } health_report { overall_health: UNKNOWN } } }
I20260809 15:39:40.252882 22836 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:39:40.249759 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51758 (local address 127.3.89.195:42417)
0809 15:39:40.249893 (+ 134us) server_negotiation.cc:206] Beginning negotiation
0809 15:39:40.249897 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 15:39:40.249911 (+ 14us) server_negotiation.cc:407] Connection header received
0809 15:39:40.250048 (+ 137us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 15:39:40.250050 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 15:39:40.250095 (+ 45us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 15:39:40.250176 (+ 81us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 15:39:40.250380 (+ 204us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:39:40.250868 (+ 488us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:39:40.251699 (+ 831us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:39:40.251920 (+ 221us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:39:40.251997 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:39:40.252229 (+ 232us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 15:39:40.252576 (+ 347us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 15:39:40.252660 (+ 84us) server_negotiation.cc:1035] Waiting for connection context
0809 15:39:40.252714 (+ 54us) server_negotiation.cc:299] Negotiation successful
0809 15:39:40.252753 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":61}
I20260809 15:39:40.257983 22625 raft_consensus.cc:1261] T 86c8bccdc6c1402fa833c810796b0276 P 0519b6b0a3ca4a138db8484f6c22828c [term 1 FOLLOWER]: Refusing update from remote peer eac1fb0c078b4f2980b8ad8ff9725d71: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 15:39:40.258000 22482 raft_consensus.cc:1261] T 86c8bccdc6c1402fa833c810796b0276 P 98843482ad594ba69390b3ce5e0e23ce [term 1 FOLLOWER]: Refusing update from remote peer eac1fb0c078b4f2980b8ad8ff9725d71: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 15:39:40.258661 22852 consensus_queue.cc:1035] T 86c8bccdc6c1402fa833c810796b0276 P eac1fb0c078b4f2980b8ad8ff9725d71 [LEADER]: Connected to new peer: Peer: permanent_uuid: "98843482ad594ba69390b3ce5e0e23ce" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 44355 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 15:39:40.258872 22849 consensus_queue.cc:1035] T 86c8bccdc6c1402fa833c810796b0276 P eac1fb0c078b4f2980b8ad8ff9725d71 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0519b6b0a3ca4a138db8484f6c22828c" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42163 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 15:39:40.273336 22326 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:53840:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 15:39:40.273489 22326 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:53840:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 15:39:40.275731 22326 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 2e607fffbe0543dc8b5b928060542e4e: Sending DeleteTablet for 3 replicas of tablet 86c8bccdc6c1402fa833c810796b0276
I20260809 15:39:40.276547 22603 tablet_service.cc:1514] Processing DeleteTablet for tablet 86c8bccdc6c1402fa833c810796b0276 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 15:39:40 UTC) from {username='slave'} at 127.0.0.1:48942
I20260809 15:39:40.276528 22461 tablet_service.cc:1514] Processing DeleteTablet for tablet 86c8bccdc6c1402fa833c810796b0276 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 15:39:40 UTC) from {username='slave'} at 127.0.0.1:58166
I20260809 15:39:40.276934 22865 tablet_replica.cc:330] T 86c8bccdc6c1402fa833c810796b0276 P 0519b6b0a3ca4a138db8484f6c22828c: stopping tablet replica
I20260809 15:39:40.277117 22747 tablet_service.cc:1514] Processing DeleteTablet for tablet 86c8bccdc6c1402fa833c810796b0276 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 15:39:40 UTC) from {username='slave'} at 127.0.0.1:51750
I20260809 15:39:40.277241 22865 raft_consensus.cc:2229] T 86c8bccdc6c1402fa833c810796b0276 P 0519b6b0a3ca4a138db8484f6c22828c [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 15:39:40.277518 22865 raft_consensus.cc:2258] T 86c8bccdc6c1402fa833c810796b0276 P 0519b6b0a3ca4a138db8484f6c22828c [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 15:39:40.277820 3431 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskm314YR/build/debug/bin/kudu with pid 22394
I20260809 15:39:40.279507 22865 ts_tablet_manager.cc:1905] T 86c8bccdc6c1402fa833c810796b0276 P 0519b6b0a3ca4a138db8484f6c22828c: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 15:39:40.284011 22867 tablet_replica.cc:330] T 86c8bccdc6c1402fa833c810796b0276 P eac1fb0c078b4f2980b8ad8ff9725d71: stopping tablet replica
I20260809 15:39:40.284082 22865 ts_tablet_manager.cc:1918] T 86c8bccdc6c1402fa833c810796b0276 P 0519b6b0a3ca4a138db8484f6c22828c: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 15:39:40.284183 22865 log.cc:1196] T 86c8bccdc6c1402fa833c810796b0276 P 0519b6b0a3ca4a138db8484f6c22828c: Deleting WAL directory at /tmp/dist-test-taskm314YR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786289732823074-3431-0/minicluster-data/ts-1/wal/wals/86c8bccdc6c1402fa833c810796b0276
I20260809 15:39:40.284504 22867 raft_consensus.cc:2229] T 86c8bccdc6c1402fa833c810796b0276 P eac1fb0c078b4f2980b8ad8ff9725d71 [term 1 LEADER]: Raft consensus shutting down.
I20260809 15:39:40.284545 22865 ts_tablet_manager.cc:1939] T 86c8bccdc6c1402fa833c810796b0276 P 0519b6b0a3ca4a138db8484f6c22828c: Deleting consensus metadata
I20260809 15:39:40.285002 22867 raft_consensus.cc:2258] T 86c8bccdc6c1402fa833c810796b0276 P eac1fb0c078b4f2980b8ad8ff9725d71 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 15:39:40.286152 22309 catalog_manager.cc:4841] TS 0519b6b0a3ca4a138db8484f6c22828c (127.3.89.194:42163): tablet 86c8bccdc6c1402fa833c810796b0276 (table test-table [id=71122d9aa0b7424386e707986aba1af9]) successfully deleted
I20260809 15:39:40.287252 22867 ts_tablet_manager.cc:1905] T 86c8bccdc6c1402fa833c810796b0276 P eac1fb0c078b4f2980b8ad8ff9725d71: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 15:39:40.290676 22867 ts_tablet_manager.cc:1918] T 86c8bccdc6c1402fa833c810796b0276 P eac1fb0c078b4f2980b8ad8ff9725d71: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 15:39:40.290778 22867 log.cc:1196] T 86c8bccdc6c1402fa833c810796b0276 P eac1fb0c078b4f2980b8ad8ff9725d71: Deleting WAL directory at /tmp/dist-test-taskm314YR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786289732823074-3431-0/minicluster-data/ts-2/wal/wals/86c8bccdc6c1402fa833c810796b0276
I20260809 15:39:40.291132 22867 ts_tablet_manager.cc:1939] T 86c8bccdc6c1402fa833c810796b0276 P eac1fb0c078b4f2980b8ad8ff9725d71: Deleting consensus metadata
W20260809 15:39:40.291723 22312 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.3.89.193:44355 (error 108)
I20260809 15:39:40.292276 22309 catalog_manager.cc:4841] TS eac1fb0c078b4f2980b8ad8ff9725d71 (127.3.89.195:42417): tablet 86c8bccdc6c1402fa833c810796b0276 (table test-table [id=71122d9aa0b7424386e707986aba1af9]) successfully deleted
I20260809 15:39:40.292619 3431 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskm314YR/build/debug/bin/kudu with pid 22539
I20260809 15:39:40.292806 22838 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:39:40.292494 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.89.193:44355 (local address 127.0.0.1:58170)
0809 15:39:40.292624 (+ 130us) negotiation.cc:107] Waiting for socket to connect
0809 15:39:40.292707 (+ 83us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.89.193:44355: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":40}
W20260809 15:39:40.294247 22312 catalog_manager.cc:4584] TS 98843482ad594ba69390b3ce5e0e23ce (127.3.89.193:44355): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 86c8bccdc6c1402fa833c810796b0276: Network error: Client connection negotiation failed: client connection to 127.3.89.193:44355: connect: Connection refused (error 111)
I20260809 15:39:40.306131 3431 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskm314YR/build/debug/bin/kudu with pid 22680
I20260809 15:39:40.313144 3431 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskm314YR/build/debug/bin/kudu with pid 22294
2026-08-09T15:39:40Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/1 (1193 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/2
Loading random data
Initializing database '/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786289732823074-3431-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 15:39:40 dist-test-slave-2w3w krb5kdc[22873](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 15:39:40 dist-test-slave-2w3w krb5kdc[22873](info): set up 2 sockets
Aug 09 15:39:40 dist-test-slave-2w3w krb5kdc[22873](info): commencing operation
krb5kdc: starting...
W20260809 15:39:41.396131 3431 mini_kdc.cc:121] Time spent starting KDC: real 1.060s user 0.003s sys 0.004s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786289732823074-3431-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786289732823074-3431-0/krb5kdc/test-user.keytab.
Aug 09 15:39:41 dist-test-slave-2w3w krb5kdc[22873](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786289981, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T15:39:41Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T15:39:41Z Disabled control of system clock
WARNING: no policy specified for kudu/127.3.89.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.3.89.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.3.89.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786289732823074-3431-0/krb5kdc/kudu_127.3.89.254.keytab.
Entry for principal kudu/127.3.89.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786289732823074-3431-0/krb5kdc/kudu_127.3.89.254.keytab.
I20260809 15:39:41.519970 3431 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskm314YR/build/debug/bin/kudu
/tmp/dist-test-taskm314YR/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786289732823074-3431-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786289732823074-3431-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786289732823074-3431-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786289732823074-3431-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.3.89.254:40777
--webserver_interface=127.3.89.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786289732823074-3431-0/krb5kdc/kudu_127.3.89.254.keytab
--principal=kudu/127.3.89.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.3.89.212:44913
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.3.89.254:40777
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786289732823074-3431-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786289732823074-3431-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786289732823074-3431-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 15:39:41.629801 22886 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 15:39:41.630048 22886 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 15:39:41.630088 22886 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 15:39:41.634030 22886 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 15:39:41.634085 22886 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 15:39:41.634104 22886 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 15:39:41.634121 22886 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 15:39:41.634136 22886 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 15:39:41.639338 22886 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.3.89.212:44913
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786289732823074-3431-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786289732823074-3431-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.3.89.254:40777
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786289732823074-3431-0/krb5kdc/kudu_127.3.89.254.keytab
--principal=kudu/127.3.89.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.3.89.254:40777
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786289732823074-3431-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.3.89.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.22886
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786289732823074-3431-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 09 Aug 2026 14:46:55 UTC on ccycloud.quasar-yswkmf.root.comops.site
build id 4576540
I20260809 15:39:41.640357 22886 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 15:39:41.641346 22886 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 15:39:41.649816 22895 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 15:39:41.649915 22886 server_base.cc:1034] running on GCE node
W20260809 15:39:41.650175 22891 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 15:39:41.650246 22892 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 15:39:41.650761 22886 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 15:39:41.651893 22886 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 15:39:41.653087 22886 hybrid_clock.cc:648] HybridClock initialized: now 1786289981653075 us; error 35 us; skew 500 ppm
Aug 09 15:39:41 dist-test-slave-2w3w krb5kdc[22873](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786289981, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.89.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 15:39:41.655802 22886 init.cc:377] Logged in from keytab as kudu/127.3.89.254@KRBTEST.COM (short username kudu)
I20260809 15:39:41.656631 22886 webserver.cc:460] Webserver started at http://127.3.89.254:35719/ using document root <none> and password file <none>
I20260809 15:39:41.657178 22886 fs_manager.cc:359] Metadata directory not provided
I20260809 15:39:41.657269 22886 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 15:39:41.657505 22886 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 15:39:41.659561 22886 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskm314YR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786289732823074-3431-0/minicluster-data/master-0/data/instance:
uuid: "28ed085824474f4ea9149b3e415b48b4"
format_stamp: "Formatted at 2026-08-09 15:39:41 on dist-test-slave-2w3w"
server_key: "ae159f976670f6954445411eef688a97"
server_key_iv: "eeb2a45010e0d6a606a03beae9a7f524"
server_key_version: "encryptionkey@0"
I20260809 15:39:41.660131 22886 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskm314YR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786289732823074-3431-0/minicluster-data/master-0/wal/instance:
uuid: "28ed085824474f4ea9149b3e415b48b4"
format_stamp: "Formatted at 2026-08-09 15:39:41 on dist-test-slave-2w3w"
server_key: "ae159f976670f6954445411eef688a97"
server_key_iv: "eeb2a45010e0d6a606a03beae9a7f524"
server_key_version: "encryptionkey@0"
I20260809 15:39:41.663777 22886 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.004s sys 0.000s
I20260809 15:39:41.666106 22904 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 15:39:41.667331 22886 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260809 15:39:41.667482 22886 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskm314YR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786289732823074-3431-0/minicluster-data/master-0/data,/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786289732823074-3431-0/minicluster-data/master-0/wal
uuid: "28ed085824474f4ea9149b3e415b48b4"
format_stamp: "Formatted at 2026-08-09 15:39:41 on dist-test-slave-2w3w"
server_key: "ae159f976670f6954445411eef688a97"
server_key_iv: "eeb2a45010e0d6a606a03beae9a7f524"
server_key_version: "encryptionkey@0"
I20260809 15:39:41.667596 22886 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskm314YR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786289732823074-3431-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskm314YR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786289732823074-3431-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskm314YR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786289732823074-3431-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 15:39:41.690862 22886 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 15:39:41.693812 22886 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 15:39:41.694031 22886 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 15:39:41.701782 22886 rpc_server.cc:307] RPC server started. Bound to: 127.3.89.254:40777
I20260809 15:39:41.701810 22962 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.3.89.254:40777 every 8 connection(s)
I20260809 15:39:41.702731 22886 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskm314YR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786289732823074-3431-0/minicluster-data/master-0/data/info.pb
I20260809 15:39:41.705363 22963 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 15:39:41.706573 3431 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskm314YR/build/debug/bin/kudu as pid 22886
I20260809 15:39:41.706743 3431 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskm314YR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786289732823074-3431-0/minicluster-data/master-0/wal/instance
I20260809 15:39:41.707151 3431 external_mini_cluster.cc:1428] Setting key 843fb5bd4c5adcbf6e6f6b34c542a0bd
I20260809 15:39:41.711408 22963 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 28ed085824474f4ea9149b3e415b48b4: Bootstrap starting.
I20260809 15:39:41.713505 22963 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 28ed085824474f4ea9149b3e415b48b4: Neither blocks nor log segments found. Creating new log.
I20260809 15:39:41.714267 22963 log.cc:824] T 00000000000000000000000000000000 P 28ed085824474f4ea9149b3e415b48b4: Log is configured to *not* fsync() on all Append() calls
Aug 09 15:39:41 dist-test-slave-2w3w krb5kdc[22873](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786289981, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.3.89.254@KRBTEST.COM
I20260809 15:39:41.716321 22963 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 28ed085824474f4ea9149b3e415b48b4: No bootstrap required, opened a new log
I20260809 15:39:41.719837 22963 raft_consensus.cc:348] T 00000000000000000000000000000000 P 28ed085824474f4ea9149b3e415b48b4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "28ed085824474f4ea9149b3e415b48b4" member_type: VOTER last_known_addr { host: "127.3.89.254" port: 40777 } }
I20260809 15:39:41.720067 22963 raft_consensus.cc:374] T 00000000000000000000000000000000 P 28ed085824474f4ea9149b3e415b48b4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 15:39:41.720126 22963 raft_consensus.cc:729] T 00000000000000000000000000000000 P 28ed085824474f4ea9149b3e415b48b4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 28ed085824474f4ea9149b3e415b48b4, State: Initialized, Role: FOLLOWER
I20260809 15:39:41.720563 22963 consensus_queue.cc:260] T 00000000000000000000000000000000 P 28ed085824474f4ea9149b3e415b48b4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "28ed085824474f4ea9149b3e415b48b4" member_type: VOTER last_known_addr { host: "127.3.89.254" port: 40777 } }
I20260809 15:39:41.720726 22963 raft_consensus.cc:388] T 00000000000000000000000000000000 P 28ed085824474f4ea9149b3e415b48b4 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 15:39:41.720810 22963 raft_consensus.cc:482] T 00000000000000000000000000000000 P 28ed085824474f4ea9149b3e415b48b4 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 15:39:41.720927 22963 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 28ed085824474f4ea9149b3e415b48b4 [term 0 FOLLOWER]: Advancing to term 1
I20260809 15:39:41.721196 22966 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:39:41.708565 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35274 (local address 127.3.89.254:40777)
0809 15:39:41.709011 (+ 446us) server_negotiation.cc:206] Beginning negotiation
0809 15:39:41.709026 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0809 15:39:41.709061 (+ 35us) server_negotiation.cc:407] Connection header received
0809 15:39:41.709689 (+ 628us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 15:39:41.709705 (+ 16us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 15:39:41.710007 (+ 302us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 15:39:41.710320 (+ 313us) server_negotiation.cc:226] Negotiated authn=SASL
0809 15:39:41.711195 (+ 875us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:39:41.712204 (+ 1009us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:39:41.713112 (+ 908us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:39:41.713620 (+ 508us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:39:41.713793 (+ 173us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:39:41.716606 (+ 2813us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 15:39:41.716610 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 15:39:41.716622 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 15:39:41.716647 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0809 15:39:41.718239 (+ 1592us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 15:39:41.718698 (+ 459us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 15:39:41.718703 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 15:39:41.718707 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 15:39:41.718815 (+ 108us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 15:39:41.719141 (+ 326us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 15:39:41.719145 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 15:39:41.719147 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 15:39:41.719526 (+ 379us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 15:39:41.719753 (+ 227us) server_negotiation.cc:1035] Waiting for connection context
0809 15:39:41.720001 (+ 248us) server_negotiation.cc:299] Negotiation successful
0809 15:39:41.720314 (+ 313us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":314,"thread_start_us":153,"threads_started":1}
I20260809 15:39:41.721858 22963 raft_consensus.cc:504] T 00000000000000000000000000000000 P 28ed085824474f4ea9149b3e415b48b4 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "28ed085824474f4ea9149b3e415b48b4" member_type: VOTER last_known_addr { host: "127.3.89.254" port: 40777 } }
I20260809 15:39:41.722169 22963 leader_election.cc:302] T 00000000000000000000000000000000 P 28ed085824474f4ea9149b3e415b48b4 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 28ed085824474f4ea9149b3e415b48b4; no voters:
I20260809 15:39:41.722488 22963 leader_election.cc:288] T 00000000000000000000000000000000 P 28ed085824474f4ea9149b3e415b48b4 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 15:39:41.722564 22968 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 28ed085824474f4ea9149b3e415b48b4 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 15:39:41.722787 22968 raft_consensus.cc:686] T 00000000000000000000000000000000 P 28ed085824474f4ea9149b3e415b48b4 [term 1 LEADER]: Becoming Leader. State: Replica: 28ed085824474f4ea9149b3e415b48b4, State: Running, Role: LEADER
I20260809 15:39:41.723095 22968 consensus_queue.cc:237] T 00000000000000000000000000000000 P 28ed085824474f4ea9149b3e415b48b4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "28ed085824474f4ea9149b3e415b48b4" member_type: VOTER last_known_addr { host: "127.3.89.254" port: 40777 } }
I20260809 15:39:41.723747 22963 sys_catalog.cc:564] T 00000000000000000000000000000000 P 28ed085824474f4ea9149b3e415b48b4 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 15:39:41.724208 22968 sys_catalog.cc:455] T 00000000000000000000000000000000 P 28ed085824474f4ea9149b3e415b48b4 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 28ed085824474f4ea9149b3e415b48b4. Latest consensus state: current_term: 1 leader_uuid: "28ed085824474f4ea9149b3e415b48b4" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "28ed085824474f4ea9149b3e415b48b4" member_type: VOTER last_known_addr { host: "127.3.89.254" port: 40777 } } }
I20260809 15:39:41.724326 22968 sys_catalog.cc:458] T 00000000000000000000000000000000 P 28ed085824474f4ea9149b3e415b48b4 [sys.catalog]: This master's current role is: LEADER
I20260809 15:39:41.725092 22969 sys_catalog.cc:455] T 00000000000000000000000000000000 P 28ed085824474f4ea9149b3e415b48b4 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "28ed085824474f4ea9149b3e415b48b4" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "28ed085824474f4ea9149b3e415b48b4" member_type: VOTER last_known_addr { host: "127.3.89.254" port: 40777 } } }
I20260809 15:39:41.725258 22969 sys_catalog.cc:458] T 00000000000000000000000000000000 P 28ed085824474f4ea9149b3e415b48b4 [sys.catalog]: This master's current role is: LEADER
I20260809 15:39:41.725600 22974 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 15:39:41.728804 22974 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 15:39:41.733829 22974 catalog_manager.cc:1347] Generated new cluster ID: 0d2e0e3beae54a9eabde6f947e062933
I20260809 15:39:41.733903 22974 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 15:39:41.745687 22974 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 15:39:41.746742 22974 catalog_manager.cc:1504] Loading token signing keys...
I20260809 15:39:41.750123 22974 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 28ed085824474f4ea9149b3e415b48b4: Generated new TSK 0
I20260809 15:39:41.750761 22974 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.3.89.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.3.89.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.3.89.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786289732823074-3431-0/krb5kdc/kudu_127.3.89.193.keytab.
Entry for principal kudu/127.3.89.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786289732823074-3431-0/krb5kdc/kudu_127.3.89.193.keytab.
I20260809 15:39:41.783937 3431 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskm314YR/build/debug/bin/kudu
/tmp/dist-test-taskm314YR/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786289732823074-3431-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786289732823074-3431-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786289732823074-3431-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786289732823074-3431-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.3.89.193:0
--local_ip_for_outbound_sockets=127.3.89.193
--webserver_interface=127.3.89.193
--webserver_port=0
--tserver_master_addrs=127.3.89.254:40777
--keytab_file=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786289732823074-3431-0/krb5kdc/kudu_127.3.89.193.keytab
--principal=kudu/127.3.89.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.3.89.212:44913
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786289732823074-3431-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786289732823074-3431-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786289732823074-3431-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 15:39:41.905306 22991 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 15:39:41.905596 22991 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 15:39:41.905692 22991 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 15:39:41.910357 22991 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 15:39:41.910451 22991 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 15:39:41.910549 22991 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.3.89.193
I20260809 15:39:41.916320 22991 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.3.89.212:44913
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786289732823074-3431-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786289732823074-3431-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786289732823074-3431-0/krb5kdc/kudu_127.3.89.193.keytab
--principal=kudu/127.3.89.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.3.89.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786289732823074-3431-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.3.89.193
--webserver_port=0
--tserver_master_addrs=127.3.89.254:40777
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.22991
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.3.89.193
--log_dir=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786289732823074-3431-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 09 Aug 2026 14:46:55 UTC on ccycloud.quasar-yswkmf.root.comops.site
build id 4576540
I20260809 15:39:41.917400 22991 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 15:39:41.918500 22991 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 15:39:41.928928 22998 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 15:39:41.929147 22991 server_base.cc:1034] running on GCE node
W20260809 15:39:41.928928 23001 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 15:39:41.929008 22999 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 15:39:41.929684 22991 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 15:39:41.930347 22991 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 15:39:41.931530 22991 hybrid_clock.cc:648] HybridClock initialized: now 1786289981931521 us; error 31 us; skew 500 ppm
Aug 09 15:39:41 dist-test-slave-2w3w krb5kdc[22873](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786289981, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.89.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 15:39:41.934387 22991 init.cc:377] Logged in from keytab as kudu/127.3.89.193@KRBTEST.COM (short username kudu)
I20260809 15:39:41.935374 22991 webserver.cc:460] Webserver started at http://127.3.89.193:41413/ using document root <none> and password file <none>
I20260809 15:39:41.935984 22991 fs_manager.cc:359] Metadata directory not provided
I20260809 15:39:41.936090 22991 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 15:39:41.936328 22991 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 15:39:41.938388 22991 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskm314YR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786289732823074-3431-0/minicluster-data/ts-0/data/instance:
uuid: "85c45510b8234215a9b4abb6cfa679ef"
format_stamp: "Formatted at 2026-08-09 15:39:41 on dist-test-slave-2w3w"
server_key: "364259d145d6fe624f447db0f608ed9f"
server_key_iv: "e08140946d053512abdb73a8c3170ef9"
server_key_version: "encryptionkey@0"
I20260809 15:39:41.939035 22991 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskm314YR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786289732823074-3431-0/minicluster-data/ts-0/wal/instance:
uuid: "85c45510b8234215a9b4abb6cfa679ef"
format_stamp: "Formatted at 2026-08-09 15:39:41 on dist-test-slave-2w3w"
server_key: "364259d145d6fe624f447db0f608ed9f"
server_key_iv: "e08140946d053512abdb73a8c3170ef9"
server_key_version: "encryptionkey@0"
I20260809 15:39:41.942637 22991 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.002s sys 0.000s
I20260809 15:39:41.945152 23008 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 15:39:41.946416 22991 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260809 15:39:41.946555 22991 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskm314YR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786289732823074-3431-0/minicluster-data/ts-0/data,/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786289732823074-3431-0/minicluster-data/ts-0/wal
uuid: "85c45510b8234215a9b4abb6cfa679ef"
format_stamp: "Formatted at 2026-08-09 15:39:41 on dist-test-slave-2w3w"
server_key: "364259d145d6fe624f447db0f608ed9f"
server_key_iv: "e08140946d053512abdb73a8c3170ef9"
server_key_version: "encryptionkey@0"
I20260809 15:39:41.946666 22991 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskm314YR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786289732823074-3431-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskm314YR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786289732823074-3431-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskm314YR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786289732823074-3431-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 15:39:41.966008 22991 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 15:39:41.969166 22991 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 15:39:41.969383 22991 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 15:39:41.969996 22991 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 15:39:41.970952 22991 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 15:39:41.971047 22991 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 15:39:41.971120 22991 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 15:39:41.971176 22991 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 15:39:41.980667 22991 rpc_server.cc:307] RPC server started. Bound to: 127.3.89.193:38793
I20260809 15:39:41.980720 23133 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.3.89.193:38793 every 8 connection(s)
I20260809 15:39:41.981590 22991 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskm314YR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786289732823074-3431-0/minicluster-data/ts-0/data/info.pb
Aug 09 15:39:41 dist-test-slave-2w3w krb5kdc[22873](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786289981, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.89.193@KRBTEST.COM for kudu/127.3.89.254@KRBTEST.COM
I20260809 15:39:41.991144 3431 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskm314YR/build/debug/bin/kudu as pid 22991
I20260809 15:39:41.991281 3431 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskm314YR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786289732823074-3431-0/minicluster-data/ts-0/wal/instance
I20260809 15:39:41.991639 3431 external_mini_cluster.cc:1428] Setting key 1c6873fb6ffcd448656e579adc22c7b5
I20260809 15:39:41.995796 22966 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:39:41.983403 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.89.193:55169 (local address 127.3.89.254:40777)
0809 15:39:41.983588 (+ 185us) server_negotiation.cc:206] Beginning negotiation
0809 15:39:41.983593 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 15:39:41.984641 (+ 1048us) server_negotiation.cc:407] Connection header received
0809 15:39:41.985516 (+ 875us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 15:39:41.985519 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 15:39:41.985570 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 15:39:41.985648 (+ 78us) server_negotiation.cc:226] Negotiated authn=SASL
0809 15:39:41.986811 (+ 1163us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:39:41.987369 (+ 558us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:39:41.988516 (+ 1147us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:39:41.988738 (+ 222us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:39:41.988817 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:39:41.991519 (+ 2702us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 15:39:41.991524 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 15:39:41.991527 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 15:39:41.991557 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0809 15:39:41.993517 (+ 1960us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 15:39:41.994061 (+ 544us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 15:39:41.994067 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 15:39:41.994072 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0809 15:39:41.994168 (+ 96us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 15:39:41.994517 (+ 349us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 15:39:41.994524 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 15:39:41.994528 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 15:39:41.994818 (+ 290us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 15:39:41.994927 (+ 109us) server_negotiation.cc:1035] Waiting for connection context
0809 15:39:41.995448 (+ 521us) server_negotiation.cc:299] Negotiation successful
0809 15:39:41.995608 (+ 160us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":82}
I20260809 15:39:41.996881 23137 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:39:41.983807 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.89.254:40777 (local address 127.3.89.193:55169)
0809 15:39:41.984489 (+ 682us) negotiation.cc:107] Waiting for socket to connect
0809 15:39:41.984531 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0809 15:39:41.985325 (+ 794us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 15:39:41.985841 (+ 516us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 15:39:41.985847 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 15:39:41.986202 (+ 355us) client_negotiation.cc:189] Negotiated authn=SASL
0809 15:39:41.986669 (+ 467us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:39:41.986682 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:39:41.987483 (+ 801us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:39:41.987485 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:39:41.988408 (+ 923us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:39:41.988418 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:39:41.988871 (+ 453us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:39:41.988875 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:39:41.989014 (+ 139us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:39:41.989660 (+ 646us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 15:39:41.989676 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0809 15:39:41.991304 (+ 1628us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 15:39:41.993636 (+ 2332us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 15:39:41.993640 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 15:39:41.993652 (+ 12us) client_negotiation.cc:760] calling sasl_client_step()
0809 15:39:41.993946 (+ 294us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 15:39:41.994273 (+ 327us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 15:39:41.994275 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 15:39:41.994276 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0809 15:39:41.994414 (+ 138us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 15:39:41.994925 (+ 511us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 15:39:41.994930 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 15:39:41.995209 (+ 279us) client_negotiation.cc:769] Sending connection context
0809 15:39:41.995370 (+ 161us) client_negotiation.cc:240] Negotiation successful
0809 15:39:41.995774 (+ 404us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":495,"thread_start_us":122,"threads_started":1}
I20260809 15:39:41.998173 23134 heartbeater.cc:347] Connected to a master server at 127.3.89.254:40777
I20260809 15:39:41.998449 23134 heartbeater.cc:464] Registering TS with master...
I20260809 15:39:41.999212 23134 heartbeater.cc:511] Master 127.3.89.254:40777 requested a full tablet report, sending...
I20260809 15:39:42.001005 22925 ts_manager.cc:194] Registered new tserver with Master: 85c45510b8234215a9b4abb6cfa679ef (127.3.89.193:38793)
I20260809 15:39:42.002652 22925 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.3.89.193@KRBTEST.COM'} at 127.3.89.193:55169
WARNING: no policy specified for kudu/127.3.89.194@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.3.89.194@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.3.89.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786289732823074-3431-0/krb5kdc/kudu_127.3.89.194.keytab.
Entry for principal kudu/127.3.89.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786289732823074-3431-0/krb5kdc/kudu_127.3.89.194.keytab.
I20260809 15:39:42.021613 3431 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskm314YR/build/debug/bin/kudu
/tmp/dist-test-taskm314YR/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786289732823074-3431-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786289732823074-3431-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786289732823074-3431-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786289732823074-3431-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.3.89.194:0
--local_ip_for_outbound_sockets=127.3.89.194
--webserver_interface=127.3.89.194
--webserver_port=0
--tserver_master_addrs=127.3.89.254:40777
--keytab_file=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786289732823074-3431-0/krb5kdc/kudu_127.3.89.194.keytab
--principal=kudu/127.3.89.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.3.89.212:44913
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786289732823074-3431-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786289732823074-3431-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786289732823074-3431-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 15:39:42.129280 23140 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 15:39:42.129503 23140 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 15:39:42.129576 23140 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 15:39:42.133556 23140 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 15:39:42.133610 23140 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 15:39:42.133663 23140 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.3.89.194
I20260809 15:39:42.138777 23140 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.3.89.212:44913
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786289732823074-3431-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786289732823074-3431-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786289732823074-3431-0/krb5kdc/kudu_127.3.89.194.keytab
--principal=kudu/127.3.89.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.3.89.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786289732823074-3431-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.3.89.194
--webserver_port=0
--tserver_master_addrs=127.3.89.254:40777
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.23140
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.3.89.194
--log_dir=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786289732823074-3431-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 09 Aug 2026 14:46:55 UTC on ccycloud.quasar-yswkmf.root.comops.site
build id 4576540
I20260809 15:39:42.139806 23140 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 15:39:42.140766 23140 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 15:39:42.150296 23149 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 15:39:42.150305 23146 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 15:39:42.150570 23140 server_base.cc:1034] running on GCE node
W20260809 15:39:42.150305 23147 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 15:39:42.151012 23140 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 15:39:42.151639 23140 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 15:39:42.152822 23140 hybrid_clock.cc:648] HybridClock initialized: now 1786289982152790 us; error 48 us; skew 500 ppm
Aug 09 15:39:42 dist-test-slave-2w3w krb5kdc[22873](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786289982, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.89.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 15:39:42.155571 23140 init.cc:377] Logged in from keytab as kudu/127.3.89.194@KRBTEST.COM (short username kudu)
I20260809 15:39:42.156379 23140 webserver.cc:460] Webserver started at http://127.3.89.194:33883/ using document root <none> and password file <none>
I20260809 15:39:42.156978 23140 fs_manager.cc:359] Metadata directory not provided
I20260809 15:39:42.157073 23140 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 15:39:42.157305 23140 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 15:39:42.159313 23140 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskm314YR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786289732823074-3431-0/minicluster-data/ts-1/data/instance:
uuid: "e0d2078e53324939be41aa7e5e0bc55e"
format_stamp: "Formatted at 2026-08-09 15:39:42 on dist-test-slave-2w3w"
server_key: "2ea02a840662b11763385e2343196acb"
server_key_iv: "1c6dee8c38af6576f1ec2ad83fd9884a"
server_key_version: "encryptionkey@0"
I20260809 15:39:42.159873 23140 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskm314YR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786289732823074-3431-0/minicluster-data/ts-1/wal/instance:
uuid: "e0d2078e53324939be41aa7e5e0bc55e"
format_stamp: "Formatted at 2026-08-09 15:39:42 on dist-test-slave-2w3w"
server_key: "2ea02a840662b11763385e2343196acb"
server_key_iv: "1c6dee8c38af6576f1ec2ad83fd9884a"
server_key_version: "encryptionkey@0"
I20260809 15:39:42.163496 23140 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.003s sys 0.000s
I20260809 15:39:42.165732 23156 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 15:39:42.166826 23140 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260809 15:39:42.167024 23140 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskm314YR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786289732823074-3431-0/minicluster-data/ts-1/data,/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786289732823074-3431-0/minicluster-data/ts-1/wal
uuid: "e0d2078e53324939be41aa7e5e0bc55e"
format_stamp: "Formatted at 2026-08-09 15:39:42 on dist-test-slave-2w3w"
server_key: "2ea02a840662b11763385e2343196acb"
server_key_iv: "1c6dee8c38af6576f1ec2ad83fd9884a"
server_key_version: "encryptionkey@0"
I20260809 15:39:42.167138 23140 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskm314YR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786289732823074-3431-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskm314YR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786289732823074-3431-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskm314YR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786289732823074-3431-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 15:39:42.187006 23140 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 15:39:42.190277 23140 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 15:39:42.190500 23140 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 15:39:42.191124 23140 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 15:39:42.191943 23140 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 15:39:42.192018 23140 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 15:39:42.192088 23140 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 15:39:42.192132 23140 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 15:39:42.201313 23140 rpc_server.cc:307] RPC server started. Bound to: 127.3.89.194:44959
I20260809 15:39:42.201339 23279 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.3.89.194:44959 every 8 connection(s)
I20260809 15:39:42.202147 23140 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskm314YR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786289732823074-3431-0/minicluster-data/ts-1/data/info.pb
I20260809 15:39:42.208510 3431 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskm314YR/build/debug/bin/kudu as pid 23140
I20260809 15:39:42.208662 3431 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskm314YR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786289732823074-3431-0/minicluster-data/ts-1/wal/instance
I20260809 15:39:42.209028 3431 external_mini_cluster.cc:1428] Setting key 048a00ae2c489b3d49127409693340e1
Aug 09 15:39:42 dist-test-slave-2w3w krb5kdc[22873](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786289982, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.89.194@KRBTEST.COM for kudu/127.3.89.254@KRBTEST.COM
I20260809 15:39:42.215207 22966 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:39:42.203907 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.89.194:56183 (local address 127.3.89.254:40777)
0809 15:39:42.204049 (+ 142us) server_negotiation.cc:206] Beginning negotiation
0809 15:39:42.204052 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0809 15:39:42.204784 (+ 732us) server_negotiation.cc:407] Connection header received
0809 15:39:42.205586 (+ 802us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 15:39:42.205589 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 15:39:42.205638 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 15:39:42.205714 (+ 76us) server_negotiation.cc:226] Negotiated authn=SASL
0809 15:39:42.206779 (+ 1065us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:39:42.207331 (+ 552us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:39:42.208592 (+ 1261us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:39:42.208920 (+ 328us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:39:42.209033 (+ 113us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:39:42.211485 (+ 2452us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 15:39:42.211492 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 15:39:42.211498 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 15:39:42.211531 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0809 15:39:42.213095 (+ 1564us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 15:39:42.213714 (+ 619us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 15:39:42.213716 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 15:39:42.213718 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 15:39:42.213790 (+ 72us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 15:39:42.214137 (+ 347us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 15:39:42.214139 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 15:39:42.214140 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 15:39:42.214304 (+ 164us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 15:39:42.214417 (+ 113us) server_negotiation.cc:1035] Waiting for connection context
0809 15:39:42.214897 (+ 480us) server_negotiation.cc:299] Negotiation successful
0809 15:39:42.215043 (+ 146us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":45}
I20260809 15:39:42.215934 23282 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:39:42.204201 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.89.254:40777 (local address 127.3.89.194:56183)
0809 15:39:42.204617 (+ 416us) negotiation.cc:107] Waiting for socket to connect
0809 15:39:42.204658 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0809 15:39:42.205392 (+ 734us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 15:39:42.205865 (+ 473us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 15:39:42.205871 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 15:39:42.206215 (+ 344us) client_negotiation.cc:189] Negotiated authn=SASL
0809 15:39:42.206652 (+ 437us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:39:42.206665 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:39:42.207437 (+ 772us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:39:42.207440 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:39:42.208459 (+ 1019us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:39:42.208471 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:39:42.209066 (+ 595us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:39:42.209069 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:39:42.209269 (+ 200us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:39:42.209811 (+ 542us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 15:39:42.209828 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0809 15:39:42.211325 (+ 1497us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 15:39:42.213214 (+ 1889us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 15:39:42.213222 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 15:39:42.213235 (+ 13us) client_negotiation.cc:760] calling sasl_client_step()
0809 15:39:42.213612 (+ 377us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 15:39:42.213883 (+ 271us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 15:39:42.213887 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 15:39:42.213891 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0809 15:39:42.214052 (+ 161us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 15:39:42.214417 (+ 365us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 15:39:42.214423 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 15:39:42.214640 (+ 217us) client_negotiation.cc:769] Sending connection context
0809 15:39:42.214841 (+ 201us) client_negotiation.cc:240] Negotiation successful
0809 15:39:42.215180 (+ 339us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":249,"thread_start_us":100,"threads_started":1}
I20260809 15:39:42.217054 23280 heartbeater.cc:347] Connected to a master server at 127.3.89.254:40777
I20260809 15:39:42.217308 23280 heartbeater.cc:464] Registering TS with master...
I20260809 15:39:42.217867 23280 heartbeater.cc:511] Master 127.3.89.254:40777 requested a full tablet report, sending...
I20260809 15:39:42.219071 22925 ts_manager.cc:194] Registered new tserver with Master: e0d2078e53324939be41aa7e5e0bc55e (127.3.89.194:44959)
I20260809 15:39:42.219872 22925 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.3.89.194@KRBTEST.COM'} at 127.3.89.194:56183
WARNING: no policy specified for kudu/127.3.89.195@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.3.89.195@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.3.89.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786289732823074-3431-0/krb5kdc/kudu_127.3.89.195.keytab.
Entry for principal kudu/127.3.89.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786289732823074-3431-0/krb5kdc/kudu_127.3.89.195.keytab.
I20260809 15:39:42.238242 3431 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskm314YR/build/debug/bin/kudu
/tmp/dist-test-taskm314YR/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786289732823074-3431-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786289732823074-3431-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786289732823074-3431-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786289732823074-3431-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.3.89.195:0
--local_ip_for_outbound_sockets=127.3.89.195
--webserver_interface=127.3.89.195
--webserver_port=0
--tserver_master_addrs=127.3.89.254:40777
--keytab_file=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786289732823074-3431-0/krb5kdc/kudu_127.3.89.195.keytab
--principal=kudu/127.3.89.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.3.89.212:44913
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786289732823074-3431-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786289732823074-3431-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786289732823074-3431-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 15:39:42.346155 23285 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 15:39:42.346381 23285 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 15:39:42.346421 23285 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 15:39:42.350499 23285 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 15:39:42.350553 23285 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 15:39:42.350658 23285 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.3.89.195
I20260809 15:39:42.355880 23285 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.3.89.212:44913
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786289732823074-3431-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786289732823074-3431-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786289732823074-3431-0/krb5kdc/kudu_127.3.89.195.keytab
--principal=kudu/127.3.89.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.3.89.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786289732823074-3431-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.3.89.195
--webserver_port=0
--tserver_master_addrs=127.3.89.254:40777
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.23285
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.3.89.195
--log_dir=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786289732823074-3431-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 09 Aug 2026 14:46:55 UTC on ccycloud.quasar-yswkmf.root.comops.site
build id 4576540
I20260809 15:39:42.356817 23285 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 15:39:42.357753 23285 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 15:39:42.366833 23290 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 15:39:42.366863 23294 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 15:39:42.366885 23285 server_base.cc:1034] running on GCE node
W20260809 15:39:42.366824 23291 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 15:39:42.367400 23285 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 15:39:42.368001 23285 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 15:39:42.369182 23285 hybrid_clock.cc:648] HybridClock initialized: now 1786289982369161 us; error 32 us; skew 500 ppm
Aug 09 15:39:42 dist-test-slave-2w3w krb5kdc[22873](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786289982, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.89.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 15:39:42.372165 23285 init.cc:377] Logged in from keytab as kudu/127.3.89.195@KRBTEST.COM (short username kudu)
I20260809 15:39:42.373016 23285 webserver.cc:460] Webserver started at http://127.3.89.195:45937/ using document root <none> and password file <none>
I20260809 15:39:42.373549 23285 fs_manager.cc:359] Metadata directory not provided
I20260809 15:39:42.373641 23285 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 15:39:42.373850 23285 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 15:39:42.375869 23285 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskm314YR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786289732823074-3431-0/minicluster-data/ts-2/data/instance:
uuid: "f99935c77f2f450f9ac25391429704b2"
format_stamp: "Formatted at 2026-08-09 15:39:42 on dist-test-slave-2w3w"
server_key: "1d31898a40556fdd7a4e152bdedf6a3f"
server_key_iv: "234dcf2c59f8fff0d1380d71d46a8818"
server_key_version: "encryptionkey@0"
I20260809 15:39:42.376423 23285 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskm314YR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786289732823074-3431-0/minicluster-data/ts-2/wal/instance:
uuid: "f99935c77f2f450f9ac25391429704b2"
format_stamp: "Formatted at 2026-08-09 15:39:42 on dist-test-slave-2w3w"
server_key: "1d31898a40556fdd7a4e152bdedf6a3f"
server_key_iv: "234dcf2c59f8fff0d1380d71d46a8818"
server_key_version: "encryptionkey@0"
I20260809 15:39:42.379904 23285 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.004s sys 0.000s
I20260809 15:39:42.382570 23303 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 15:39:42.383764 23285 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.000s sys 0.003s
I20260809 15:39:42.383904 23285 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskm314YR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786289732823074-3431-0/minicluster-data/ts-2/data,/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786289732823074-3431-0/minicluster-data/ts-2/wal
uuid: "f99935c77f2f450f9ac25391429704b2"
format_stamp: "Formatted at 2026-08-09 15:39:42 on dist-test-slave-2w3w"
server_key: "1d31898a40556fdd7a4e152bdedf6a3f"
server_key_iv: "234dcf2c59f8fff0d1380d71d46a8818"
server_key_version: "encryptionkey@0"
I20260809 15:39:42.384020 23285 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskm314YR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786289732823074-3431-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskm314YR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786289732823074-3431-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskm314YR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786289732823074-3431-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 15:39:42.406545 23285 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 15:39:42.409925 23285 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 15:39:42.410137 23285 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 15:39:42.410729 23285 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 15:39:42.411559 23285 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 15:39:42.411631 23285 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 15:39:42.411690 23285 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 15:39:42.411742 23285 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 15:39:42.420547 23285 rpc_server.cc:307] RPC server started. Bound to: 127.3.89.195:34709
I20260809 15:39:42.420617 23424 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.3.89.195:34709 every 8 connection(s)
I20260809 15:39:42.421408 23285 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskm314YR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786289732823074-3431-0/minicluster-data/ts-2/data/info.pb
I20260809 15:39:42.424690 3431 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskm314YR/build/debug/bin/kudu as pid 23285
I20260809 15:39:42.424834 3431 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskm314YR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786289732823074-3431-0/minicluster-data/ts-2/wal/instance
I20260809 15:39:42.425179 3431 external_mini_cluster.cc:1428] Setting key 371ba3a06a7f45f750643f01f4f54015
Aug 09 15:39:42 dist-test-slave-2w3w krb5kdc[22873](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786289982, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.89.195@KRBTEST.COM for kudu/127.3.89.254@KRBTEST.COM
I20260809 15:39:42.434502 22966 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:39:42.423202 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.89.195:54435 (local address 127.3.89.254:40777)
0809 15:39:42.423346 (+ 144us) server_negotiation.cc:206] Beginning negotiation
0809 15:39:42.423350 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 15:39:42.424444 (+ 1094us) server_negotiation.cc:407] Connection header received
0809 15:39:42.425629 (+ 1185us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 15:39:42.425636 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 15:39:42.425726 (+ 90us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 15:39:42.425818 (+ 92us) server_negotiation.cc:226] Negotiated authn=SASL
0809 15:39:42.427190 (+ 1372us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:39:42.427804 (+ 614us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:39:42.428905 (+ 1101us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:39:42.429110 (+ 205us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:39:42.429185 (+ 75us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:39:42.431499 (+ 2314us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 15:39:42.431502 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 15:39:42.431504 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 15:39:42.431529 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0809 15:39:42.432807 (+ 1278us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 15:39:42.433262 (+ 455us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 15:39:42.433264 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 15:39:42.433266 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 15:39:42.433338 (+ 72us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 15:39:42.433624 (+ 286us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 15:39:42.433626 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 15:39:42.433627 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 15:39:42.433789 (+ 162us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 15:39:42.433861 (+ 72us) server_negotiation.cc:1035] Waiting for connection context
0809 15:39:42.434289 (+ 428us) server_negotiation.cc:299] Negotiation successful
0809 15:39:42.434383 (+ 94us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":55}
I20260809 15:39:42.435381 23427 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:39:42.423582 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.89.254:40777 (local address 127.3.89.195:54435)
0809 15:39:42.424266 (+ 684us) negotiation.cc:107] Waiting for socket to connect
0809 15:39:42.424319 (+ 53us) client_negotiation.cc:174] Beginning negotiation
0809 15:39:42.425391 (+ 1072us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 15:39:42.426003 (+ 612us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 15:39:42.426010 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 15:39:42.426442 (+ 432us) client_negotiation.cc:189] Negotiated authn=SASL
0809 15:39:42.427052 (+ 610us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:39:42.427066 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:39:42.427919 (+ 853us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:39:42.427922 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:39:42.428797 (+ 875us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:39:42.428805 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:39:42.429232 (+ 427us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:39:42.429235 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:39:42.429388 (+ 153us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:39:42.429909 (+ 521us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 15:39:42.429924 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0809 15:39:42.431349 (+ 1425us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 15:39:42.432909 (+ 1560us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 15:39:42.432913 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 15:39:42.432924 (+ 11us) client_negotiation.cc:760] calling sasl_client_step()
0809 15:39:42.433168 (+ 244us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 15:39:42.433414 (+ 246us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 15:39:42.433416 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 15:39:42.433417 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0809 15:39:42.433541 (+ 124us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 15:39:42.433872 (+ 331us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 15:39:42.433876 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 15:39:42.434087 (+ 211us) client_negotiation.cc:769] Sending connection context
0809 15:39:42.434227 (+ 140us) client_negotiation.cc:240] Negotiation successful
0809 15:39:42.434459 (+ 232us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":513,"thread_start_us":111,"threads_started":1}
I20260809 15:39:42.436472 23425 heartbeater.cc:347] Connected to a master server at 127.3.89.254:40777
I20260809 15:39:42.436717 23425 heartbeater.cc:464] Registering TS with master...
I20260809 15:39:42.437245 23425 heartbeater.cc:511] Master 127.3.89.254:40777 requested a full tablet report, sending...
I20260809 15:39:42.438381 22925 ts_manager.cc:194] Registered new tserver with Master: f99935c77f2f450f9ac25391429704b2 (127.3.89.195:34709)
I20260809 15:39:42.439033 22925 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.3.89.195@KRBTEST.COM'} at 127.3.89.195:54435
I20260809 15:39:42.439388 3431 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260809 15:39:42.449183 22966 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:39:42.441413 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35276 (local address 127.3.89.254:40777)
0809 15:39:42.441555 (+ 142us) server_negotiation.cc:206] Beginning negotiation
0809 15:39:42.441559 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 15:39:42.441722 (+ 163us) server_negotiation.cc:407] Connection header received
0809 15:39:42.441843 (+ 121us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 15:39:42.441847 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 15:39:42.441886 (+ 39us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 15:39:42.441980 (+ 94us) server_negotiation.cc:226] Negotiated authn=SASL
0809 15:39:42.442696 (+ 716us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:39:42.443275 (+ 579us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:39:42.444135 (+ 860us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:39:42.444365 (+ 230us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:39:42.444478 (+ 113us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:39:42.445987 (+ 1509us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 15:39:42.445991 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 15:39:42.445994 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 15:39:42.446018 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0809 15:39:42.447338 (+ 1320us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 15:39:42.447848 (+ 510us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 15:39:42.447852 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 15:39:42.447854 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 15:39:42.447936 (+ 82us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 15:39:42.448281 (+ 345us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 15:39:42.448285 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 15:39:42.448286 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 15:39:42.448486 (+ 200us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 15:39:42.448651 (+ 165us) server_negotiation.cc:1035] Waiting for connection context
0809 15:39:42.448927 (+ 276us) server_negotiation.cc:299] Negotiation successful
0809 15:39:42.449017 (+ 90us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":63}
I20260809 15:39:42.451834 22925 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:35276:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260809 15:39:42.454885 22925 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260809 15:39:42.468844 23437 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:39:42.463026 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.89.195:34709 (local address 127.0.0.1:44030)
0809 15:39:42.464000 (+ 974us) negotiation.cc:107] Waiting for socket to connect
0809 15:39:42.464016 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0809 15:39:42.464096 (+ 80us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 15:39:42.464595 (+ 499us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 15:39:42.464602 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 15:39:42.464628 (+ 26us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 15:39:42.464773 (+ 145us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:39:42.464791 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:39:42.465965 (+ 1174us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:39:42.465970 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:39:42.467408 (+ 1438us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:39:42.467420 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:39:42.468281 (+ 861us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:39:42.468285 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:39:42.468392 (+ 107us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:39:42.468426 (+ 34us) client_negotiation.cc:769] Sending connection context
0809 15:39:42.468568 (+ 142us) client_negotiation.cc:240] Negotiation successful
0809 15:39:42.468684 (+ 116us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":899,"mutex_wait_us":5,"thread_start_us":90,"threads_started":1}
I20260809 15:39:42.468968 23440 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:39:42.463674 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.89.193:38793 (local address 127.0.0.1:56222)
0809 15:39:42.464268 (+ 594us) negotiation.cc:107] Waiting for socket to connect
0809 15:39:42.464280 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0809 15:39:42.464347 (+ 67us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 15:39:42.465012 (+ 665us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 15:39:42.465014 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 15:39:42.465031 (+ 17us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 15:39:42.465104 (+ 73us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:39:42.465110 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:39:42.466380 (+ 1270us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:39:42.466385 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:39:42.467555 (+ 1170us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:39:42.467566 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:39:42.468566 (+ 1000us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:39:42.468570 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:39:42.468653 (+ 83us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:39:42.468673 (+ 20us) client_negotiation.cc:769] Sending connection context
0809 15:39:42.468740 (+ 67us) client_negotiation.cc:240] Negotiation successful
0809 15:39:42.468794 (+ 54us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":541,"mutex_wait_us":40,"thread_start_us":98,"threads_started":1}
I20260809 15:39:42.469304 23436 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:39:42.462922 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.89.194:44959 (local address 127.0.0.1:48078)
0809 15:39:42.463376 (+ 454us) negotiation.cc:107] Waiting for socket to connect
0809 15:39:42.463427 (+ 51us) client_negotiation.cc:174] Beginning negotiation
0809 15:39:42.463942 (+ 515us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 15:39:42.464702 (+ 760us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 15:39:42.464705 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 15:39:42.464718 (+ 13us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 15:39:42.464788 (+ 70us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:39:42.464793 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:39:42.466254 (+ 1461us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:39:42.466257 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:39:42.467437 (+ 1180us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:39:42.467444 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:39:42.468706 (+ 1262us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:39:42.468712 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:39:42.468813 (+ 101us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:39:42.468835 (+ 22us) client_negotiation.cc:769] Sending connection context
0809 15:39:42.468909 (+ 74us) client_negotiation.cc:240] Negotiation successful
0809 15:39:42.469175 (+ 266us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":287,"mutex_wait_us":45,"thread_start_us":80,"threads_started":1}
I20260809 15:39:42.469316 23442 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:39:42.463699 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56222 (local address 127.3.89.193:38793)
0809 15:39:42.464596 (+ 897us) server_negotiation.cc:206] Beginning negotiation
0809 15:39:42.464616 (+ 20us) server_negotiation.cc:399] Waiting for connection header
0809 15:39:42.464638 (+ 22us) server_negotiation.cc:407] Connection header received
0809 15:39:42.464720 (+ 82us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 15:39:42.464723 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 15:39:42.464869 (+ 146us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 15:39:42.465034 (+ 165us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 15:39:42.465188 (+ 154us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:39:42.466245 (+ 1057us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:39:42.467683 (+ 1438us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:39:42.468449 (+ 766us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:39:42.468546 (+ 97us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:39:42.468660 (+ 114us) server_negotiation.cc:1035] Waiting for connection context
0809 15:39:42.469096 (+ 436us) server_negotiation.cc:299] Negotiation successful
0809 15:39:42.469208 (+ 112us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":795,"thread_start_us":122,"threads_started":1}
I20260809 15:39:42.469467 23441 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:39:42.463686 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44030 (local address 127.3.89.195:34709)
0809 15:39:42.464104 (+ 418us) server_negotiation.cc:206] Beginning negotiation
0809 15:39:42.464116 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0809 15:39:42.464139 (+ 23us) server_negotiation.cc:407] Connection header received
0809 15:39:42.464213 (+ 74us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 15:39:42.464220 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 15:39:42.464413 (+ 193us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 15:39:42.464939 (+ 526us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 15:39:42.465099 (+ 160us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:39:42.465838 (+ 739us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:39:42.467540 (+ 1702us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:39:42.468146 (+ 606us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:39:42.468281 (+ 135us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:39:42.468423 (+ 142us) server_negotiation.cc:1035] Waiting for connection context
0809 15:39:42.468749 (+ 326us) server_negotiation.cc:299] Negotiation successful
0809 15:39:42.468841 (+ 92us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":317,"thread_start_us":134,"threads_started":1}
I20260809 15:39:42.469844 23438 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:39:42.463094 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48078 (local address 127.3.89.194:44959)
0809 15:39:42.463821 (+ 727us) server_negotiation.cc:206] Beginning negotiation
0809 15:39:42.463833 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0809 15:39:42.463857 (+ 24us) server_negotiation.cc:407] Connection header received
0809 15:39:42.464165 (+ 308us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 15:39:42.464174 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 15:39:42.464384 (+ 210us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 15:39:42.464544 (+ 160us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 15:39:42.464914 (+ 370us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:39:42.466110 (+ 1196us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:39:42.467569 (+ 1459us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:39:42.468379 (+ 810us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:39:42.468497 (+ 118us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:39:42.468638 (+ 141us) server_negotiation.cc:1035] Waiting for connection context
0809 15:39:42.469580 (+ 942us) server_negotiation.cc:299] Negotiation successful
0809 15:39:42.469679 (+ 99us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":608,"thread_start_us":141,"threads_started":1}
I20260809 15:39:42.472342 23062 tablet_service.cc:1467] Processing CreateTablet for tablet 8af614944e5241bfa597cc13bcc5bba1 (DEFAULT_TABLE table=test-table [id=1c0fada712a147238f8690df94f699a1]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 15:39:42.472361 23355 tablet_service.cc:1467] Processing CreateTablet for tablet 8af614944e5241bfa597cc13bcc5bba1 (DEFAULT_TABLE table=test-table [id=1c0fada712a147238f8690df94f699a1]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 15:39:42.472361 23207 tablet_service.cc:1467] Processing CreateTablet for tablet 8af614944e5241bfa597cc13bcc5bba1 (DEFAULT_TABLE table=test-table [id=1c0fada712a147238f8690df94f699a1]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 15:39:42.473330 23062 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 8af614944e5241bfa597cc13bcc5bba1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 15:39:42.473354 23207 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 8af614944e5241bfa597cc13bcc5bba1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 15:39:42.473361 23355 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 8af614944e5241bfa597cc13bcc5bba1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 15:39:42.480710 23445 tablet_bootstrap.cc:492] T 8af614944e5241bfa597cc13bcc5bba1 P e0d2078e53324939be41aa7e5e0bc55e: Bootstrap starting.
I20260809 15:39:42.480710 23444 tablet_bootstrap.cc:492] T 8af614944e5241bfa597cc13bcc5bba1 P f99935c77f2f450f9ac25391429704b2: Bootstrap starting.
I20260809 15:39:42.480787 23443 tablet_bootstrap.cc:492] T 8af614944e5241bfa597cc13bcc5bba1 P 85c45510b8234215a9b4abb6cfa679ef: Bootstrap starting.
I20260809 15:39:42.482779 23443 tablet_bootstrap.cc:654] T 8af614944e5241bfa597cc13bcc5bba1 P 85c45510b8234215a9b4abb6cfa679ef: Neither blocks nor log segments found. Creating new log.
I20260809 15:39:42.482878 23444 tablet_bootstrap.cc:654] T 8af614944e5241bfa597cc13bcc5bba1 P f99935c77f2f450f9ac25391429704b2: Neither blocks nor log segments found. Creating new log.
I20260809 15:39:42.483349 23445 tablet_bootstrap.cc:654] T 8af614944e5241bfa597cc13bcc5bba1 P e0d2078e53324939be41aa7e5e0bc55e: Neither blocks nor log segments found. Creating new log.
I20260809 15:39:42.483969 23443 log.cc:824] T 8af614944e5241bfa597cc13bcc5bba1 P 85c45510b8234215a9b4abb6cfa679ef: Log is configured to *not* fsync() on all Append() calls
I20260809 15:39:42.484165 23444 log.cc:824] T 8af614944e5241bfa597cc13bcc5bba1 P f99935c77f2f450f9ac25391429704b2: Log is configured to *not* fsync() on all Append() calls
I20260809 15:39:42.484165 23445 log.cc:824] T 8af614944e5241bfa597cc13bcc5bba1 P e0d2078e53324939be41aa7e5e0bc55e: Log is configured to *not* fsync() on all Append() calls
I20260809 15:39:42.486020 23443 tablet_bootstrap.cc:492] T 8af614944e5241bfa597cc13bcc5bba1 P 85c45510b8234215a9b4abb6cfa679ef: No bootstrap required, opened a new log
I20260809 15:39:42.486020 23445 tablet_bootstrap.cc:492] T 8af614944e5241bfa597cc13bcc5bba1 P e0d2078e53324939be41aa7e5e0bc55e: No bootstrap required, opened a new log
I20260809 15:39:42.486055 23444 tablet_bootstrap.cc:492] T 8af614944e5241bfa597cc13bcc5bba1 P f99935c77f2f450f9ac25391429704b2: No bootstrap required, opened a new log
I20260809 15:39:42.486205 23445 ts_tablet_manager.cc:1397] T 8af614944e5241bfa597cc13bcc5bba1 P e0d2078e53324939be41aa7e5e0bc55e: Time spent bootstrapping tablet: real 0.006s user 0.004s sys 0.000s
I20260809 15:39:42.486205 23443 ts_tablet_manager.cc:1397] T 8af614944e5241bfa597cc13bcc5bba1 P 85c45510b8234215a9b4abb6cfa679ef: Time spent bootstrapping tablet: real 0.007s user 0.004s sys 0.000s
I20260809 15:39:42.486227 23444 ts_tablet_manager.cc:1397] T 8af614944e5241bfa597cc13bcc5bba1 P f99935c77f2f450f9ac25391429704b2: Time spent bootstrapping tablet: real 0.006s user 0.004s sys 0.000s
I20260809 15:39:42.489452 23444 raft_consensus.cc:348] T 8af614944e5241bfa597cc13bcc5bba1 P f99935c77f2f450f9ac25391429704b2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e0d2078e53324939be41aa7e5e0bc55e" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 44959 } } peers { permanent_uuid: "f99935c77f2f450f9ac25391429704b2" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 34709 } } peers { permanent_uuid: "85c45510b8234215a9b4abb6cfa679ef" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 38793 } }
I20260809 15:39:42.489706 23444 raft_consensus.cc:374] T 8af614944e5241bfa597cc13bcc5bba1 P f99935c77f2f450f9ac25391429704b2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 15:39:42.489768 23444 raft_consensus.cc:729] T 8af614944e5241bfa597cc13bcc5bba1 P f99935c77f2f450f9ac25391429704b2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f99935c77f2f450f9ac25391429704b2, State: Initialized, Role: FOLLOWER
I20260809 15:39:42.490070 23443 raft_consensus.cc:348] T 8af614944e5241bfa597cc13bcc5bba1 P 85c45510b8234215a9b4abb6cfa679ef [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e0d2078e53324939be41aa7e5e0bc55e" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 44959 } } peers { permanent_uuid: "f99935c77f2f450f9ac25391429704b2" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 34709 } } peers { permanent_uuid: "85c45510b8234215a9b4abb6cfa679ef" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 38793 } }
I20260809 15:39:42.490070 23445 raft_consensus.cc:348] T 8af614944e5241bfa597cc13bcc5bba1 P e0d2078e53324939be41aa7e5e0bc55e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e0d2078e53324939be41aa7e5e0bc55e" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 44959 } } peers { permanent_uuid: "f99935c77f2f450f9ac25391429704b2" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 34709 } } peers { permanent_uuid: "85c45510b8234215a9b4abb6cfa679ef" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 38793 } }
I20260809 15:39:42.490216 23444 consensus_queue.cc:260] T 8af614944e5241bfa597cc13bcc5bba1 P f99935c77f2f450f9ac25391429704b2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e0d2078e53324939be41aa7e5e0bc55e" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 44959 } } peers { permanent_uuid: "f99935c77f2f450f9ac25391429704b2" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 34709 } } peers { permanent_uuid: "85c45510b8234215a9b4abb6cfa679ef" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 38793 } }
I20260809 15:39:42.490321 23443 raft_consensus.cc:374] T 8af614944e5241bfa597cc13bcc5bba1 P 85c45510b8234215a9b4abb6cfa679ef [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 15:39:42.490321 23445 raft_consensus.cc:374] T 8af614944e5241bfa597cc13bcc5bba1 P e0d2078e53324939be41aa7e5e0bc55e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 15:39:42.490381 23445 raft_consensus.cc:729] T 8af614944e5241bfa597cc13bcc5bba1 P e0d2078e53324939be41aa7e5e0bc55e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e0d2078e53324939be41aa7e5e0bc55e, State: Initialized, Role: FOLLOWER
I20260809 15:39:42.490381 23443 raft_consensus.cc:729] T 8af614944e5241bfa597cc13bcc5bba1 P 85c45510b8234215a9b4abb6cfa679ef [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 85c45510b8234215a9b4abb6cfa679ef, State: Initialized, Role: FOLLOWER
I20260809 15:39:42.490840 23425 heartbeater.cc:503] Master 127.3.89.254:40777 was elected leader, sending a full tablet report...
I20260809 15:39:42.490849 23443 consensus_queue.cc:260] T 8af614944e5241bfa597cc13bcc5bba1 P 85c45510b8234215a9b4abb6cfa679ef [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e0d2078e53324939be41aa7e5e0bc55e" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 44959 } } peers { permanent_uuid: "f99935c77f2f450f9ac25391429704b2" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 34709 } } peers { permanent_uuid: "85c45510b8234215a9b4abb6cfa679ef" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 38793 } }
I20260809 15:39:42.491261 23444 ts_tablet_manager.cc:1428] T 8af614944e5241bfa597cc13bcc5bba1 P f99935c77f2f450f9ac25391429704b2: Time spent starting tablet: real 0.005s user 0.006s sys 0.000s
I20260809 15:39:42.491480 23445 consensus_queue.cc:260] T 8af614944e5241bfa597cc13bcc5bba1 P e0d2078e53324939be41aa7e5e0bc55e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e0d2078e53324939be41aa7e5e0bc55e" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 44959 } } peers { permanent_uuid: "f99935c77f2f450f9ac25391429704b2" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 34709 } } peers { permanent_uuid: "85c45510b8234215a9b4abb6cfa679ef" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 38793 } }
I20260809 15:39:42.491586 23134 heartbeater.cc:503] Master 127.3.89.254:40777 was elected leader, sending a full tablet report...
I20260809 15:39:42.491830 23443 ts_tablet_manager.cc:1428] T 8af614944e5241bfa597cc13bcc5bba1 P 85c45510b8234215a9b4abb6cfa679ef: Time spent starting tablet: real 0.006s user 0.003s sys 0.004s
I20260809 15:39:42.492300 23445 ts_tablet_manager.cc:1428] T 8af614944e5241bfa597cc13bcc5bba1 P e0d2078e53324939be41aa7e5e0bc55e: Time spent starting tablet: real 0.006s user 0.006s sys 0.000s
I20260809 15:39:42.492645 23280 heartbeater.cc:503] Master 127.3.89.254:40777 was elected leader, sending a full tablet report...
I20260809 15:39:42.493944 23451 raft_consensus.cc:482] T 8af614944e5241bfa597cc13bcc5bba1 P 85c45510b8234215a9b4abb6cfa679ef [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 15:39:42.494120 23451 raft_consensus.cc:504] T 8af614944e5241bfa597cc13bcc5bba1 P 85c45510b8234215a9b4abb6cfa679ef [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e0d2078e53324939be41aa7e5e0bc55e" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 44959 } } peers { permanent_uuid: "f99935c77f2f450f9ac25391429704b2" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 34709 } } peers { permanent_uuid: "85c45510b8234215a9b4abb6cfa679ef" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 38793 } }
I20260809 15:39:42.495739 23451 leader_election.cc:288] T 8af614944e5241bfa597cc13bcc5bba1 P 85c45510b8234215a9b4abb6cfa679ef [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e0d2078e53324939be41aa7e5e0bc55e (127.3.89.194:44959), f99935c77f2f450f9ac25391429704b2 (127.3.89.195:34709)
I20260809 15:39:42.499305 23137 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:39:42.495455 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.89.194:44959 (local address 127.3.89.193:49655)
0809 15:39:42.495610 (+ 155us) negotiation.cc:107] Waiting for socket to connect
0809 15:39:42.495627 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0809 15:39:42.495714 (+ 87us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 15:39:42.496138 (+ 424us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 15:39:42.496141 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 15:39:42.496162 (+ 21us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 15:39:42.496295 (+ 133us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:39:42.496303 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:39:42.497082 (+ 779us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:39:42.497085 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:39:42.498142 (+ 1057us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:39:42.498152 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:39:42.498812 (+ 660us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:39:42.498815 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:39:42.498928 (+ 113us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:39:42.499001 (+ 73us) client_negotiation.cc:769] Sending connection context
0809 15:39:42.499072 (+ 71us) client_negotiation.cc:240] Negotiation successful
0809 15:39:42.499150 (+ 78us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":60}
I20260809 15:39:42.499323 23438 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:39:42.495753 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.89.193:49655 (local address 127.3.89.194:44959)
0809 15:39:42.495877 (+ 124us) server_negotiation.cc:206] Beginning negotiation
0809 15:39:42.495882 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 15:39:42.495894 (+ 12us) server_negotiation.cc:407] Connection header received
0809 15:39:42.495932 (+ 38us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 15:39:42.495935 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 15:39:42.495980 (+ 45us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 15:39:42.496065 (+ 85us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 15:39:42.496424 (+ 359us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:39:42.496979 (+ 555us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:39:42.498262 (+ 1283us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:39:42.498676 (+ 414us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:39:42.498817 (+ 141us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:39:42.498867 (+ 50us) server_negotiation.cc:1035] Waiting for connection context
0809 15:39:42.499098 (+ 231us) server_negotiation.cc:299] Negotiation successful
0809 15:39:42.499232 (+ 134us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":60}
I20260809 15:39:42.499825 23453 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:39:42.496106 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.89.195:34709 (local address 127.3.89.193:58209)
0809 15:39:42.496410 (+ 304us) negotiation.cc:107] Waiting for socket to connect
0809 15:39:42.496552 (+ 142us) client_negotiation.cc:174] Beginning negotiation
0809 15:39:42.496671 (+ 119us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 15:39:42.496915 (+ 244us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 15:39:42.496917 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 15:39:42.496933 (+ 16us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 15:39:42.497062 (+ 129us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:39:42.497071 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:39:42.497714 (+ 643us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:39:42.497717 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:39:42.498646 (+ 929us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:39:42.498654 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:39:42.499389 (+ 735us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:39:42.499393 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:39:42.499488 (+ 95us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:39:42.499509 (+ 21us) client_negotiation.cc:769] Sending connection context
0809 15:39:42.499608 (+ 99us) client_negotiation.cc:240] Negotiation successful
0809 15:39:42.499673 (+ 65us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":234,"thread_start_us":97,"threads_started":1}
I20260809 15:39:42.499835 23232 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8af614944e5241bfa597cc13bcc5bba1" candidate_uuid: "85c45510b8234215a9b4abb6cfa679ef" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e0d2078e53324939be41aa7e5e0bc55e" is_pre_election: true
I20260809 15:39:42.499930 23441 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:39:42.496363 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.89.193:58209 (local address 127.3.89.195:34709)
0809 15:39:42.496464 (+ 101us) server_negotiation.cc:206] Beginning negotiation
0809 15:39:42.496468 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 15:39:42.496633 (+ 165us) server_negotiation.cc:407] Connection header received
0809 15:39:42.496776 (+ 143us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 15:39:42.496779 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 15:39:42.496818 (+ 39us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 15:39:42.496905 (+ 87us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 15:39:42.497156 (+ 251us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:39:42.497610 (+ 454us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:39:42.498761 (+ 1151us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:39:42.499249 (+ 488us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:39:42.499387 (+ 138us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:39:42.499442 (+ 55us) server_negotiation.cc:1035] Waiting for connection context
0809 15:39:42.499741 (+ 299us) server_negotiation.cc:299] Negotiation successful
0809 15:39:42.499807 (+ 66us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":42}
I20260809 15:39:42.500123 23232 raft_consensus.cc:2454] T 8af614944e5241bfa597cc13bcc5bba1 P e0d2078e53324939be41aa7e5e0bc55e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 85c45510b8234215a9b4abb6cfa679ef in term 0.
I20260809 15:39:42.500411 23377 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8af614944e5241bfa597cc13bcc5bba1" candidate_uuid: "85c45510b8234215a9b4abb6cfa679ef" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f99935c77f2f450f9ac25391429704b2" is_pre_election: true
I20260809 15:39:42.500633 23012 leader_election.cc:302] T 8af614944e5241bfa597cc13bcc5bba1 P 85c45510b8234215a9b4abb6cfa679ef [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 85c45510b8234215a9b4abb6cfa679ef, e0d2078e53324939be41aa7e5e0bc55e; no voters:
I20260809 15:39:42.500689 23377 raft_consensus.cc:2454] T 8af614944e5241bfa597cc13bcc5bba1 P f99935c77f2f450f9ac25391429704b2 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 85c45510b8234215a9b4abb6cfa679ef in term 0.
I20260809 15:39:42.500890 23451 raft_consensus.cc:2781] T 8af614944e5241bfa597cc13bcc5bba1 P 85c45510b8234215a9b4abb6cfa679ef [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260809 15:39:42.501008 23451 raft_consensus.cc:482] T 8af614944e5241bfa597cc13bcc5bba1 P 85c45510b8234215a9b4abb6cfa679ef [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 15:39:42.501096 23451 raft_consensus.cc:3037] T 8af614944e5241bfa597cc13bcc5bba1 P 85c45510b8234215a9b4abb6cfa679ef [term 0 FOLLOWER]: Advancing to term 1
I20260809 15:39:42.502311 23451 raft_consensus.cc:504] T 8af614944e5241bfa597cc13bcc5bba1 P 85c45510b8234215a9b4abb6cfa679ef [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e0d2078e53324939be41aa7e5e0bc55e" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 44959 } } peers { permanent_uuid: "f99935c77f2f450f9ac25391429704b2" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 34709 } } peers { permanent_uuid: "85c45510b8234215a9b4abb6cfa679ef" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 38793 } }
I20260809 15:39:42.502671 23451 leader_election.cc:288] T 8af614944e5241bfa597cc13bcc5bba1 P 85c45510b8234215a9b4abb6cfa679ef [CANDIDATE]: Term 1 election: Requested vote from peers e0d2078e53324939be41aa7e5e0bc55e (127.3.89.194:44959), f99935c77f2f450f9ac25391429704b2 (127.3.89.195:34709)
I20260809 15:39:42.503005 23232 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8af614944e5241bfa597cc13bcc5bba1" candidate_uuid: "85c45510b8234215a9b4abb6cfa679ef" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e0d2078e53324939be41aa7e5e0bc55e"
I20260809 15:39:42.503111 23377 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8af614944e5241bfa597cc13bcc5bba1" candidate_uuid: "85c45510b8234215a9b4abb6cfa679ef" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f99935c77f2f450f9ac25391429704b2"
I20260809 15:39:42.503139 23232 raft_consensus.cc:3037] T 8af614944e5241bfa597cc13bcc5bba1 P e0d2078e53324939be41aa7e5e0bc55e [term 0 FOLLOWER]: Advancing to term 1
I20260809 15:39:42.503224 23377 raft_consensus.cc:3037] T 8af614944e5241bfa597cc13bcc5bba1 P f99935c77f2f450f9ac25391429704b2 [term 0 FOLLOWER]: Advancing to term 1
I20260809 15:39:42.504134 23232 raft_consensus.cc:2454] T 8af614944e5241bfa597cc13bcc5bba1 P e0d2078e53324939be41aa7e5e0bc55e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 85c45510b8234215a9b4abb6cfa679ef in term 1.
I20260809 15:39:42.504241 23377 raft_consensus.cc:2454] T 8af614944e5241bfa597cc13bcc5bba1 P f99935c77f2f450f9ac25391429704b2 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 85c45510b8234215a9b4abb6cfa679ef in term 1.
I20260809 15:39:42.504501 23012 leader_election.cc:302] T 8af614944e5241bfa597cc13bcc5bba1 P 85c45510b8234215a9b4abb6cfa679ef [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 85c45510b8234215a9b4abb6cfa679ef, e0d2078e53324939be41aa7e5e0bc55e; no voters:
I20260809 15:39:42.504675 23451 raft_consensus.cc:2781] T 8af614944e5241bfa597cc13bcc5bba1 P 85c45510b8234215a9b4abb6cfa679ef [term 1 FOLLOWER]: Leader election won for term 1
I20260809 15:39:42.504911 23451 raft_consensus.cc:686] T 8af614944e5241bfa597cc13bcc5bba1 P 85c45510b8234215a9b4abb6cfa679ef [term 1 LEADER]: Becoming Leader. State: Replica: 85c45510b8234215a9b4abb6cfa679ef, State: Running, Role: LEADER
I20260809 15:39:42.505205 23451 consensus_queue.cc:237] T 8af614944e5241bfa597cc13bcc5bba1 P 85c45510b8234215a9b4abb6cfa679ef [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e0d2078e53324939be41aa7e5e0bc55e" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 44959 } } peers { permanent_uuid: "f99935c77f2f450f9ac25391429704b2" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 34709 } } peers { permanent_uuid: "85c45510b8234215a9b4abb6cfa679ef" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 38793 } }
I20260809 15:39:42.508086 22925 catalog_manager.cc:5495] T 8af614944e5241bfa597cc13bcc5bba1 P 85c45510b8234215a9b4abb6cfa679ef reported cstate change: term changed from 0 to 1, leader changed from <none> to 85c45510b8234215a9b4abb6cfa679ef (127.3.89.193). New cstate: current_term: 1 leader_uuid: "85c45510b8234215a9b4abb6cfa679ef" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e0d2078e53324939be41aa7e5e0bc55e" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 44959 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f99935c77f2f450f9ac25391429704b2" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 34709 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "85c45510b8234215a9b4abb6cfa679ef" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 38793 } health_report { overall_health: HEALTHY } } }
I20260809 15:39:42.519207 23442 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:39:42.516014 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56230 (local address 127.3.89.193:38793)
0809 15:39:42.516126 (+ 112us) server_negotiation.cc:206] Beginning negotiation
0809 15:39:42.516130 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 15:39:42.516222 (+ 92us) server_negotiation.cc:407] Connection header received
0809 15:39:42.516348 (+ 126us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 15:39:42.516350 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 15:39:42.516397 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 15:39:42.516488 (+ 91us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 15:39:42.516686 (+ 198us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:39:42.517152 (+ 466us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:39:42.517937 (+ 785us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:39:42.518136 (+ 199us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:39:42.518209 (+ 73us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:39:42.518480 (+ 271us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 15:39:42.518821 (+ 341us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 15:39:42.518925 (+ 104us) server_negotiation.cc:1035] Waiting for connection context
0809 15:39:42.519085 (+ 160us) server_negotiation.cc:299] Negotiation successful
0809 15:39:42.519116 (+ 31us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":43}
I20260809 15:39:42.524487 23377 raft_consensus.cc:1261] T 8af614944e5241bfa597cc13bcc5bba1 P f99935c77f2f450f9ac25391429704b2 [term 1 FOLLOWER]: Refusing update from remote peer 85c45510b8234215a9b4abb6cfa679ef: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 15:39:42.524485 23232 raft_consensus.cc:1261] T 8af614944e5241bfa597cc13bcc5bba1 P e0d2078e53324939be41aa7e5e0bc55e [term 1 FOLLOWER]: Refusing update from remote peer 85c45510b8234215a9b4abb6cfa679ef: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 15:39:42.525195 23454 consensus_queue.cc:1035] T 8af614944e5241bfa597cc13bcc5bba1 P 85c45510b8234215a9b4abb6cfa679ef [LEADER]: Connected to new peer: Peer: permanent_uuid: "e0d2078e53324939be41aa7e5e0bc55e" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 44959 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 15:39:42.525460 23451 consensus_queue.cc:1035] T 8af614944e5241bfa597cc13bcc5bba1 P 85c45510b8234215a9b4abb6cfa679ef [LEADER]: Connected to new peer: Peer: permanent_uuid: "f99935c77f2f450f9ac25391429704b2" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 34709 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 15:39:42.542342 22925 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:35276:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 15:39:42.542536 22925 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:35276:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 15:39:42.545433 22925 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 28ed085824474f4ea9149b3e415b48b4: Sending DeleteTablet for 3 replicas of tablet 8af614944e5241bfa597cc13bcc5bba1
I20260809 15:39:42.546200 23355 tablet_service.cc:1514] Processing DeleteTablet for tablet 8af614944e5241bfa597cc13bcc5bba1 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 15:39:42 UTC) from {username='kudu', principal='kudu/127.3.89.254@KRBTEST.COM'} at 127.0.0.1:44030
I20260809 15:39:42.546200 23207 tablet_service.cc:1514] Processing DeleteTablet for tablet 8af614944e5241bfa597cc13bcc5bba1 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 15:39:42 UTC) from {username='kudu', principal='kudu/127.3.89.254@KRBTEST.COM'} at 127.0.0.1:48078
I20260809 15:39:42.546396 23062 tablet_service.cc:1514] Processing DeleteTablet for tablet 8af614944e5241bfa597cc13bcc5bba1 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 15:39:42 UTC) from {username='kudu', principal='kudu/127.3.89.254@KRBTEST.COM'} at 127.0.0.1:56222
I20260809 15:39:42.546725 23467 tablet_replica.cc:330] T 8af614944e5241bfa597cc13bcc5bba1 P f99935c77f2f450f9ac25391429704b2: stopping tablet replica
I20260809 15:39:42.547067 23467 raft_consensus.cc:2229] T 8af614944e5241bfa597cc13bcc5bba1 P f99935c77f2f450f9ac25391429704b2 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 15:39:42.547080 23470 tablet_replica.cc:330] T 8af614944e5241bfa597cc13bcc5bba1 P 85c45510b8234215a9b4abb6cfa679ef: stopping tablet replica
I20260809 15:39:42.547374 23467 raft_consensus.cc:2258] T 8af614944e5241bfa597cc13bcc5bba1 P f99935c77f2f450f9ac25391429704b2 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 15:39:42.547374 23470 raft_consensus.cc:2229] T 8af614944e5241bfa597cc13bcc5bba1 P 85c45510b8234215a9b4abb6cfa679ef [term 1 LEADER]: Raft consensus shutting down.
I20260809 15:39:42.547605 3431 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskm314YR/build/debug/bin/kudu with pid 22991
I20260809 15:39:42.547765 23469 tablet_replica.cc:330] T 8af614944e5241bfa597cc13bcc5bba1 P e0d2078e53324939be41aa7e5e0bc55e: stopping tablet replica
I20260809 15:39:42.548525 23469 raft_consensus.cc:2229] T 8af614944e5241bfa597cc13bcc5bba1 P e0d2078e53324939be41aa7e5e0bc55e [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 15:39:42.548758 23469 raft_consensus.cc:2258] T 8af614944e5241bfa597cc13bcc5bba1 P e0d2078e53324939be41aa7e5e0bc55e [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 15:39:42.550292 23469 ts_tablet_manager.cc:1905] T 8af614944e5241bfa597cc13bcc5bba1 P e0d2078e53324939be41aa7e5e0bc55e: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 15:39:42.550318 23467 ts_tablet_manager.cc:1905] T 8af614944e5241bfa597cc13bcc5bba1 P f99935c77f2f450f9ac25391429704b2: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 15:39:42.552665 23469 ts_tablet_manager.cc:1918] T 8af614944e5241bfa597cc13bcc5bba1 P e0d2078e53324939be41aa7e5e0bc55e: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 15:39:42.552829 23469 log.cc:1196] T 8af614944e5241bfa597cc13bcc5bba1 P e0d2078e53324939be41aa7e5e0bc55e: Deleting WAL directory at /tmp/dist-test-taskm314YR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786289732823074-3431-0/minicluster-data/ts-1/wal/wals/8af614944e5241bfa597cc13bcc5bba1
I20260809 15:39:42.553154 23469 ts_tablet_manager.cc:1939] T 8af614944e5241bfa597cc13bcc5bba1 P e0d2078e53324939be41aa7e5e0bc55e: Deleting consensus metadata
I20260809 15:39:42.553681 23467 ts_tablet_manager.cc:1918] T 8af614944e5241bfa597cc13bcc5bba1 P f99935c77f2f450f9ac25391429704b2: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 15:39:42.553785 23467 log.cc:1196] T 8af614944e5241bfa597cc13bcc5bba1 P f99935c77f2f450f9ac25391429704b2: Deleting WAL directory at /tmp/dist-test-taskm314YR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786289732823074-3431-0/minicluster-data/ts-2/wal/wals/8af614944e5241bfa597cc13bcc5bba1
I20260809 15:39:42.554131 23467 ts_tablet_manager.cc:1939] T 8af614944e5241bfa597cc13bcc5bba1 P f99935c77f2f450f9ac25391429704b2: Deleting consensus metadata
I20260809 15:39:42.554301 22908 catalog_manager.cc:4841] TS e0d2078e53324939be41aa7e5e0bc55e (127.3.89.194:44959): tablet 8af614944e5241bfa597cc13bcc5bba1 (table test-table [id=1c0fada712a147238f8690df94f699a1]) successfully deleted
I20260809 15:39:42.555290 22905 catalog_manager.cc:4841] TS f99935c77f2f450f9ac25391429704b2 (127.3.89.195:34709): tablet 8af614944e5241bfa597cc13bcc5bba1 (table test-table [id=1c0fada712a147238f8690df94f699a1]) successfully deleted
W20260809 15:39:42.556882 22906 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.3.89.193:38793 (error 108)
I20260809 15:39:42.557086 3431 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskm314YR/build/debug/bin/kudu with pid 23140
I20260809 15:39:42.559190 23436 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:39:42.558751 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.89.193:38793 (local address 127.0.0.1:56232)
0809 15:39:42.559006 (+ 255us) negotiation.cc:107] Waiting for socket to connect
0809 15:39:42.559086 (+ 80us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.89.193:38793: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":128}
W20260809 15:39:42.559357 22906 catalog_manager.cc:4584] TS 85c45510b8234215a9b4abb6cfa679ef (127.3.89.193:38793): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 8af614944e5241bfa597cc13bcc5bba1: Network error: Client connection negotiation failed: client connection to 127.3.89.193:38793: connect: Connection refused (error 111)
I20260809 15:39:42.565318 3431 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskm314YR/build/debug/bin/kudu with pid 23285
I20260809 15:39:42.572388 3431 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskm314YR/build/debug/bin/kudu with pid 22886
2026-08-09T15:39:42Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/2 (2260 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/3
2026-08-09T15:39:42Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T15:39:42Z Disabled control of system clock
I20260809 15:39:42.605998 3431 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskm314YR/build/debug/bin/kudu
/tmp/dist-test-taskm314YR/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786289732823074-3431-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786289732823074-3431-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786289732823074-3431-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786289732823074-3431-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.3.89.254:40071
--webserver_interface=127.3.89.254
--webserver_port=0
--builtin_ntp_servers=127.3.89.212:39977
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.3.89.254:40071
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260809 15:39:42.715724 23476 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 15:39:42.715998 23476 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 15:39:42.716034 23476 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 15:39:42.719971 23476 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 15:39:42.720021 23476 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 15:39:42.720036 23476 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 15:39:42.720052 23476 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 15:39:42.720067 23476 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 15:39:42.724835 23476 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.3.89.212:39977
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786289732823074-3431-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786289732823074-3431-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.3.89.254:40071
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.3.89.254:40071
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786289732823074-3431-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.3.89.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.23476
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786289732823074-3431-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 09 Aug 2026 14:46:55 UTC on ccycloud.quasar-yswkmf.root.comops.site
build id 4576540
I20260809 15:39:42.725829 23476 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 15:39:42.727152 23476 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 15:39:42.735227 23483 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 15:39:42.735270 23486 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 15:39:42.735359 23476 server_base.cc:1034] running on GCE node
W20260809 15:39:42.735242 23482 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 15:39:42.736049 23476 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 15:39:42.736995 23476 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 15:39:42.738188 23476 hybrid_clock.cc:648] HybridClock initialized: now 1786289982738158 us; error 42 us; skew 500 ppm
I20260809 15:39:42.739986 23476 webserver.cc:460] Webserver started at http://127.3.89.254:43163/ using document root <none> and password file <none>
I20260809 15:39:42.740538 23476 fs_manager.cc:359] Metadata directory not provided
I20260809 15:39:42.740631 23476 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 15:39:42.740888 23476 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 15:39:42.742919 23476 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskm314YR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786289732823074-3431-0/minicluster-data/master-0/data/instance:
uuid: "bf24fedf691d442c8cf055457a36c8dc"
format_stamp: "Formatted at 2026-08-09 15:39:42 on dist-test-slave-2w3w"
server_key: "b8dd2c9d3cdcd266e14287f7ada6b36e"
server_key_iv: "52650b200a70d39f3a0e7045f7a5a55c"
server_key_version: "encryptionkey@0"
I20260809 15:39:42.743537 23476 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskm314YR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786289732823074-3431-0/minicluster-data/master-0/wal/instance:
uuid: "bf24fedf691d442c8cf055457a36c8dc"
format_stamp: "Formatted at 2026-08-09 15:39:42 on dist-test-slave-2w3w"
server_key: "b8dd2c9d3cdcd266e14287f7ada6b36e"
server_key_iv: "52650b200a70d39f3a0e7045f7a5a55c"
server_key_version: "encryptionkey@0"
I20260809 15:39:42.747109 23476 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.001s sys 0.003s
I20260809 15:39:42.749395 23493 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 15:39:42.750555 23476 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260809 15:39:42.750689 23476 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskm314YR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786289732823074-3431-0/minicluster-data/master-0/data,/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786289732823074-3431-0/minicluster-data/master-0/wal
uuid: "bf24fedf691d442c8cf055457a36c8dc"
format_stamp: "Formatted at 2026-08-09 15:39:42 on dist-test-slave-2w3w"
server_key: "b8dd2c9d3cdcd266e14287f7ada6b36e"
server_key_iv: "52650b200a70d39f3a0e7045f7a5a55c"
server_key_version: "encryptionkey@0"
I20260809 15:39:42.750815 23476 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskm314YR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786289732823074-3431-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskm314YR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786289732823074-3431-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskm314YR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786289732823074-3431-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 15:39:42.773159 23476 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 15:39:42.773923 23476 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 15:39:42.774128 23476 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 15:39:42.781257 23476 rpc_server.cc:307] RPC server started. Bound to: 127.3.89.254:40071
I20260809 15:39:42.781270 23551 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.3.89.254:40071 every 8 connection(s)
I20260809 15:39:42.782193 23476 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskm314YR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786289732823074-3431-0/minicluster-data/master-0/data/info.pb
I20260809 15:39:42.785224 23552 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 15:39:42.790549 23552 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P bf24fedf691d442c8cf055457a36c8dc: Bootstrap starting.
I20260809 15:39:42.792055 3431 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskm314YR/build/debug/bin/kudu as pid 23476
I20260809 15:39:42.792166 3431 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskm314YR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786289732823074-3431-0/minicluster-data/master-0/wal/instance
I20260809 15:39:42.792435 3431 external_mini_cluster.cc:1428] Setting key 92f706b716f6f84ccb68addd878c9944
I20260809 15:39:42.793161 23552 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P bf24fedf691d442c8cf055457a36c8dc: Neither blocks nor log segments found. Creating new log.
I20260809 15:39:42.793948 23552 log.cc:824] T 00000000000000000000000000000000 P bf24fedf691d442c8cf055457a36c8dc: Log is configured to *not* fsync() on all Append() calls
I20260809 15:39:42.796097 23552 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P bf24fedf691d442c8cf055457a36c8dc: No bootstrap required, opened a new log
I20260809 15:39:42.800257 23552 raft_consensus.cc:348] T 00000000000000000000000000000000 P bf24fedf691d442c8cf055457a36c8dc [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bf24fedf691d442c8cf055457a36c8dc" member_type: VOTER last_known_addr { host: "127.3.89.254" port: 40071 } }
I20260809 15:39:42.800457 23552 raft_consensus.cc:374] T 00000000000000000000000000000000 P bf24fedf691d442c8cf055457a36c8dc [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 15:39:42.800539 23552 raft_consensus.cc:729] T 00000000000000000000000000000000 P bf24fedf691d442c8cf055457a36c8dc [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bf24fedf691d442c8cf055457a36c8dc, State: Initialized, Role: FOLLOWER
I20260809 15:39:42.800997 23552 consensus_queue.cc:260] T 00000000000000000000000000000000 P bf24fedf691d442c8cf055457a36c8dc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bf24fedf691d442c8cf055457a36c8dc" member_type: VOTER last_known_addr { host: "127.3.89.254" port: 40071 } }
I20260809 15:39:42.801127 23552 raft_consensus.cc:388] T 00000000000000000000000000000000 P bf24fedf691d442c8cf055457a36c8dc [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 15:39:42.801213 23552 raft_consensus.cc:482] T 00000000000000000000000000000000 P bf24fedf691d442c8cf055457a36c8dc [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 15:39:42.801368 23552 raft_consensus.cc:3037] T 00000000000000000000000000000000 P bf24fedf691d442c8cf055457a36c8dc [term 0 FOLLOWER]: Advancing to term 1
I20260809 15:39:42.801851 23555 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:39:42.793631 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50764 (local address 127.3.89.254:40071)
0809 15:39:42.794284 (+ 653us) server_negotiation.cc:206] Beginning negotiation
0809 15:39:42.794318 (+ 34us) server_negotiation.cc:399] Waiting for connection header
0809 15:39:42.794364 (+ 46us) server_negotiation.cc:407] Connection header received
0809 15:39:42.795212 (+ 848us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 15:39:42.795232 (+ 20us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 15:39:42.795528 (+ 296us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 15:39:42.795800 (+ 272us) server_negotiation.cc:226] Negotiated authn=SASL
0809 15:39:42.796223 (+ 423us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:39:42.797184 (+ 961us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:39:42.798433 (+ 1249us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:39:42.798877 (+ 444us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:39:42.799058 (+ 181us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:39:42.799981 (+ 923us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 15:39:42.799985 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 15:39:42.799996 (+ 11us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 15:39:42.800349 (+ 353us) server_negotiation.cc:922] calling sasl_server_start()
0809 15:39:42.800402 (+ 53us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 15:39:42.800419 (+ 17us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 15:39:42.800538 (+ 119us) server_negotiation.cc:1035] Waiting for connection context
0809 15:39:42.800704 (+ 166us) server_negotiation.cc:299] Negotiation successful
0809 15:39:42.800989 (+ 285us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":285,"thread_start_us":144,"threads_started":1}
I20260809 15:39:42.802284 23552 raft_consensus.cc:504] T 00000000000000000000000000000000 P bf24fedf691d442c8cf055457a36c8dc [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bf24fedf691d442c8cf055457a36c8dc" member_type: VOTER last_known_addr { host: "127.3.89.254" port: 40071 } }
I20260809 15:39:42.802613 23552 leader_election.cc:302] T 00000000000000000000000000000000 P bf24fedf691d442c8cf055457a36c8dc [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: bf24fedf691d442c8cf055457a36c8dc; no voters:
I20260809 15:39:42.802929 23552 leader_election.cc:288] T 00000000000000000000000000000000 P bf24fedf691d442c8cf055457a36c8dc [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 15:39:42.803027 23558 raft_consensus.cc:2781] T 00000000000000000000000000000000 P bf24fedf691d442c8cf055457a36c8dc [term 1 FOLLOWER]: Leader election won for term 1
I20260809 15:39:42.803212 23558 raft_consensus.cc:686] T 00000000000000000000000000000000 P bf24fedf691d442c8cf055457a36c8dc [term 1 LEADER]: Becoming Leader. State: Replica: bf24fedf691d442c8cf055457a36c8dc, State: Running, Role: LEADER
I20260809 15:39:42.803488 23558 consensus_queue.cc:237] T 00000000000000000000000000000000 P bf24fedf691d442c8cf055457a36c8dc [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bf24fedf691d442c8cf055457a36c8dc" member_type: VOTER last_known_addr { host: "127.3.89.254" port: 40071 } }
I20260809 15:39:42.804150 23552 sys_catalog.cc:564] T 00000000000000000000000000000000 P bf24fedf691d442c8cf055457a36c8dc [sys.catalog]: configured and running, proceeding with master startup.
I20260809 15:39:42.805124 23559 sys_catalog.cc:455] T 00000000000000000000000000000000 P bf24fedf691d442c8cf055457a36c8dc [sys.catalog]: SysCatalogTable state changed. Reason: New leader bf24fedf691d442c8cf055457a36c8dc. Latest consensus state: current_term: 1 leader_uuid: "bf24fedf691d442c8cf055457a36c8dc" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bf24fedf691d442c8cf055457a36c8dc" member_type: VOTER last_known_addr { host: "127.3.89.254" port: 40071 } } }
I20260809 15:39:42.805151 23560 sys_catalog.cc:455] T 00000000000000000000000000000000 P bf24fedf691d442c8cf055457a36c8dc [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "bf24fedf691d442c8cf055457a36c8dc" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bf24fedf691d442c8cf055457a36c8dc" member_type: VOTER last_known_addr { host: "127.3.89.254" port: 40071 } } }
I20260809 15:39:42.805244 23559 sys_catalog.cc:458] T 00000000000000000000000000000000 P bf24fedf691d442c8cf055457a36c8dc [sys.catalog]: This master's current role is: LEADER
I20260809 15:39:42.805261 23560 sys_catalog.cc:458] T 00000000000000000000000000000000 P bf24fedf691d442c8cf055457a36c8dc [sys.catalog]: This master's current role is: LEADER
I20260809 15:39:42.805662 23568 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 15:39:42.808692 23568 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 15:39:42.814096 23568 catalog_manager.cc:1347] Generated new cluster ID: c5834f7de75f459ea9afe74582d61efa
I20260809 15:39:42.814155 23568 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 15:39:42.837984 23568 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 15:39:42.839233 23568 catalog_manager.cc:1504] Loading token signing keys...
I20260809 15:39:42.847570 23568 catalog_manager.cc:5868] T 00000000000000000000000000000000 P bf24fedf691d442c8cf055457a36c8dc: Generated new TSK 0
I20260809 15:39:42.848342 23568 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260809 15:39:42.852882 3431 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskm314YR/build/debug/bin/kudu
/tmp/dist-test-taskm314YR/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786289732823074-3431-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786289732823074-3431-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786289732823074-3431-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786289732823074-3431-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.3.89.193:0
--local_ip_for_outbound_sockets=127.3.89.193
--webserver_interface=127.3.89.193
--webserver_port=0
--tserver_master_addrs=127.3.89.254:40071
--builtin_ntp_servers=127.3.89.212:39977
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260809 15:39:42.965463 23580 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 15:39:42.965714 23580 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 15:39:42.965778 23580 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 15:39:42.969864 23580 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 15:39:42.969923 23580 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 15:39:42.969975 23580 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.3.89.193
I20260809 15:39:42.974777 23580 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.3.89.212:39977
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786289732823074-3431-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786289732823074-3431-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.3.89.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786289732823074-3431-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.3.89.193
--webserver_port=0
--tserver_master_addrs=127.3.89.254:40071
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.23580
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.3.89.193
--log_dir=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786289732823074-3431-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 09 Aug 2026 14:46:55 UTC on ccycloud.quasar-yswkmf.root.comops.site
build id 4576540
I20260809 15:39:42.975895 23580 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 15:39:42.977264 23580 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 15:39:42.987010 23585 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 15:39:42.987010 23588 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 15:39:42.987214 23580 server_base.cc:1034] running on GCE node
W20260809 15:39:42.987072 23586 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 15:39:42.987696 23580 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 15:39:42.988315 23580 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 15:39:42.989491 23580 hybrid_clock.cc:648] HybridClock initialized: now 1786289982989476 us; error 25 us; skew 500 ppm
I20260809 15:39:42.991400 23580 webserver.cc:460] Webserver started at http://127.3.89.193:43739/ using document root <none> and password file <none>
I20260809 15:39:42.991984 23580 fs_manager.cc:359] Metadata directory not provided
I20260809 15:39:42.992046 23580 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 15:39:42.992293 23580 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 15:39:42.994313 23580 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskm314YR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786289732823074-3431-0/minicluster-data/ts-0/data/instance:
uuid: "8460063038f14ab5898b25822335d44a"
format_stamp: "Formatted at 2026-08-09 15:39:42 on dist-test-slave-2w3w"
server_key: "af5dcd5c7141798e133d77bf20891c2c"
server_key_iv: "f293c9dc2ca3db618afeb23561301274"
server_key_version: "encryptionkey@0"
I20260809 15:39:42.994879 23580 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskm314YR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786289732823074-3431-0/minicluster-data/ts-0/wal/instance:
uuid: "8460063038f14ab5898b25822335d44a"
format_stamp: "Formatted at 2026-08-09 15:39:42 on dist-test-slave-2w3w"
server_key: "af5dcd5c7141798e133d77bf20891c2c"
server_key_iv: "f293c9dc2ca3db618afeb23561301274"
server_key_version: "encryptionkey@0"
I20260809 15:39:42.998633 23580 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.003s sys 0.000s
I20260809 15:39:43.001282 23596 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 15:39:43.002434 23580 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.000s sys 0.002s
I20260809 15:39:43.002578 23580 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskm314YR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786289732823074-3431-0/minicluster-data/ts-0/data,/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786289732823074-3431-0/minicluster-data/ts-0/wal
uuid: "8460063038f14ab5898b25822335d44a"
format_stamp: "Formatted at 2026-08-09 15:39:42 on dist-test-slave-2w3w"
server_key: "af5dcd5c7141798e133d77bf20891c2c"
server_key_iv: "f293c9dc2ca3db618afeb23561301274"
server_key_version: "encryptionkey@0"
I20260809 15:39:43.002692 23580 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskm314YR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786289732823074-3431-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskm314YR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786289732823074-3431-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskm314YR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786289732823074-3431-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 15:39:43.025902 23580 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 15:39:43.026605 23580 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 15:39:43.026844 23580 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 15:39:43.027475 23580 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 15:39:43.028424 23580 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 15:39:43.028496 23580 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 15:39:43.028568 23580 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 15:39:43.028620 23580 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 15:39:43.037354 23580 rpc_server.cc:307] RPC server started. Bound to: 127.3.89.193:41675
I20260809 15:39:43.037384 23721 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.3.89.193:41675 every 8 connection(s)
I20260809 15:39:43.038259 23580 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskm314YR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786289732823074-3431-0/minicluster-data/ts-0/data/info.pb
I20260809 15:39:43.040437 3431 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskm314YR/build/debug/bin/kudu as pid 23580
I20260809 15:39:43.040570 3431 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskm314YR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786289732823074-3431-0/minicluster-data/ts-0/wal/instance
I20260809 15:39:43.040926 3431 external_mini_cluster.cc:1428] Setting key 8577e7765b6b53a439175d950aa33606
I20260809 15:39:43.043073 3431 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskm314YR/build/debug/bin/kudu
/tmp/dist-test-taskm314YR/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786289732823074-3431-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786289732823074-3431-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786289732823074-3431-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786289732823074-3431-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.3.89.194:0
--local_ip_for_outbound_sockets=127.3.89.194
--webserver_interface=127.3.89.194
--webserver_port=0
--tserver_master_addrs=127.3.89.254:40071
--builtin_ntp_servers=127.3.89.212:39977
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260809 15:39:43.048734 23555 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:39:43.040062 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.89.193:54453 (local address 127.3.89.254:40071)
0809 15:39:43.040217 (+ 155us) server_negotiation.cc:206] Beginning negotiation
0809 15:39:43.040221 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 15:39:43.041428 (+ 1207us) server_negotiation.cc:407] Connection header received
0809 15:39:43.043183 (+ 1755us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 15:39:43.043187 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 15:39:43.043241 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 15:39:43.043329 (+ 88us) server_negotiation.cc:226] Negotiated authn=SASL
0809 15:39:43.044200 (+ 871us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:39:43.045096 (+ 896us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:39:43.046271 (+ 1175us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:39:43.046530 (+ 259us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:39:43.046647 (+ 117us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:39:43.047965 (+ 1318us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 15:39:43.047969 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 15:39:43.047973 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 15:39:43.047991 (+ 18us) server_negotiation.cc:922] calling sasl_server_start()
0809 15:39:43.048016 (+ 25us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 15:39:43.048030 (+ 14us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 15:39:43.048124 (+ 94us) server_negotiation.cc:1035] Waiting for connection context
0809 15:39:43.048335 (+ 211us) server_negotiation.cc:299] Negotiation successful
0809 15:39:43.048440 (+ 105us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":48}
I20260809 15:39:43.051247 23724 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:39:43.040475 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.89.254:40071 (local address 127.3.89.193:54453)
0809 15:39:43.041247 (+ 772us) negotiation.cc:107] Waiting for socket to connect
0809 15:39:43.041289 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0809 15:39:43.042922 (+ 1633us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 15:39:43.043501 (+ 579us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 15:39:43.043509 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 15:39:43.043591 (+ 82us) client_negotiation.cc:189] Negotiated authn=SASL
0809 15:39:43.044054 (+ 463us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:39:43.044065 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:39:43.045257 (+ 1192us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:39:43.045265 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:39:43.046138 (+ 873us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:39:43.046148 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:39:43.046686 (+ 538us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:39:43.046690 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:39:43.046858 (+ 168us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:39:43.047560 (+ 702us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0809 15:39:43.047563 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0809 15:39:43.047623 (+ 60us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0809 15:39:43.047706 (+ 83us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 15:39:43.048139 (+ 433us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 15:39:43.048150 (+ 11us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 15:39:43.048159 (+ 9us) client_negotiation.cc:769] Sending connection context
0809 15:39:43.048308 (+ 149us) client_negotiation.cc:240] Negotiation successful
0809 15:39:43.048619 (+ 311us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":338,"thread_start_us":171,"threads_started":1}
I20260809 15:39:43.052917 23722 heartbeater.cc:347] Connected to a master server at 127.3.89.254:40071
I20260809 15:39:43.053161 23722 heartbeater.cc:464] Registering TS with master...
I20260809 15:39:43.053654 23722 heartbeater.cc:511] Master 127.3.89.254:40071 requested a full tablet report, sending...
I20260809 15:39:43.055197 23513 ts_manager.cc:194] Registered new tserver with Master: 8460063038f14ab5898b25822335d44a (127.3.89.193:41675)
I20260809 15:39:43.056309 23513 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.3.89.193:54453
W20260809 15:39:43.158794 23725 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 15:39:43.159081 23725 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 15:39:43.159162 23725 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 15:39:43.163105 23725 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 15:39:43.163180 23725 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 15:39:43.163266 23725 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.3.89.194
I20260809 15:39:43.168455 23725 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.3.89.212:39977
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786289732823074-3431-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786289732823074-3431-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.3.89.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786289732823074-3431-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.3.89.194
--webserver_port=0
--tserver_master_addrs=127.3.89.254:40071
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.23725
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.3.89.194
--log_dir=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786289732823074-3431-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 09 Aug 2026 14:46:55 UTC on ccycloud.quasar-yswkmf.root.comops.site
build id 4576540
I20260809 15:39:43.169456 23725 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 15:39:43.170760 23725 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 15:39:43.179772 23734 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 15:39:43.179814 23730 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 15:39:43.179848 23731 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 15:39:43.179981 23725 server_base.cc:1034] running on GCE node
I20260809 15:39:43.180380 23725 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 15:39:43.180971 23725 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 15:39:43.182126 23725 hybrid_clock.cc:648] HybridClock initialized: now 1786289983182101 us; error 54 us; skew 500 ppm
I20260809 15:39:43.183969 23725 webserver.cc:460] Webserver started at http://127.3.89.194:41423/ using document root <none> and password file <none>
I20260809 15:39:43.184518 23725 fs_manager.cc:359] Metadata directory not provided
I20260809 15:39:43.184584 23725 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 15:39:43.184788 23725 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 15:39:43.186745 23725 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskm314YR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786289732823074-3431-0/minicluster-data/ts-1/data/instance:
uuid: "8d1218ae5ef5440a89f06ac7d82572a4"
format_stamp: "Formatted at 2026-08-09 15:39:43 on dist-test-slave-2w3w"
server_key: "ebe025993e89ed52dfa39f1453587fa8"
server_key_iv: "729e7d02641545f5837056156a170f68"
server_key_version: "encryptionkey@0"
I20260809 15:39:43.187369 23725 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskm314YR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786289732823074-3431-0/minicluster-data/ts-1/wal/instance:
uuid: "8d1218ae5ef5440a89f06ac7d82572a4"
format_stamp: "Formatted at 2026-08-09 15:39:43 on dist-test-slave-2w3w"
server_key: "ebe025993e89ed52dfa39f1453587fa8"
server_key_iv: "729e7d02641545f5837056156a170f68"
server_key_version: "encryptionkey@0"
I20260809 15:39:43.190797 23725 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.003s sys 0.000s
I20260809 15:39:43.193184 23742 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 15:39:43.194371 23725 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260809 15:39:43.194509 23725 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskm314YR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786289732823074-3431-0/minicluster-data/ts-1/data,/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786289732823074-3431-0/minicluster-data/ts-1/wal
uuid: "8d1218ae5ef5440a89f06ac7d82572a4"
format_stamp: "Formatted at 2026-08-09 15:39:43 on dist-test-slave-2w3w"
server_key: "ebe025993e89ed52dfa39f1453587fa8"
server_key_iv: "729e7d02641545f5837056156a170f68"
server_key_version: "encryptionkey@0"
I20260809 15:39:43.194621 23725 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskm314YR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786289732823074-3431-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskm314YR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786289732823074-3431-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskm314YR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786289732823074-3431-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 15:39:43.207918 23725 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 15:39:43.208531 23725 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 15:39:43.208724 23725 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 15:39:43.209283 23725 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 15:39:43.210256 23725 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 15:39:43.210328 23725 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 15:39:43.210397 23725 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 15:39:43.210443 23725 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 15:39:43.219291 23725 rpc_server.cc:307] RPC server started. Bound to: 127.3.89.194:42561
I20260809 15:39:43.219323 23867 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.3.89.194:42561 every 8 connection(s)
I20260809 15:39:43.220129 23725 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskm314YR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786289732823074-3431-0/minicluster-data/ts-1/data/info.pb
I20260809 15:39:43.224764 3431 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskm314YR/build/debug/bin/kudu as pid 23725
I20260809 15:39:43.224915 3431 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskm314YR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786289732823074-3431-0/minicluster-data/ts-1/wal/instance
I20260809 15:39:43.225208 3431 external_mini_cluster.cc:1428] Setting key c1ca0fb314a3c778f589b53e79725582
I20260809 15:39:43.227465 3431 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskm314YR/build/debug/bin/kudu
/tmp/dist-test-taskm314YR/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786289732823074-3431-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786289732823074-3431-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786289732823074-3431-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786289732823074-3431-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.3.89.195:0
--local_ip_for_outbound_sockets=127.3.89.195
--webserver_interface=127.3.89.195
--webserver_port=0
--tserver_master_addrs=127.3.89.254:40071
--builtin_ntp_servers=127.3.89.212:39977
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260809 15:39:43.229442 23555 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:39:43.221790 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.89.194:48809 (local address 127.3.89.254:40071)
0809 15:39:43.221929 (+ 139us) server_negotiation.cc:206] Beginning negotiation
0809 15:39:43.221933 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 15:39:43.222901 (+ 968us) server_negotiation.cc:407] Connection header received
0809 15:39:43.224296 (+ 1395us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 15:39:43.224300 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 15:39:43.224357 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 15:39:43.224434 (+ 77us) server_negotiation.cc:226] Negotiated authn=SASL
0809 15:39:43.225384 (+ 950us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:39:43.226090 (+ 706us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:39:43.227345 (+ 1255us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:39:43.227708 (+ 363us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:39:43.227837 (+ 129us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:39:43.228876 (+ 1039us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 15:39:43.228882 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 15:39:43.228888 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 15:39:43.228906 (+ 18us) server_negotiation.cc:922] calling sasl_server_start()
0809 15:39:43.228932 (+ 26us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 15:39:43.228945 (+ 13us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 15:39:43.229025 (+ 80us) server_negotiation.cc:1035] Waiting for connection context
0809 15:39:43.229165 (+ 140us) server_negotiation.cc:299] Negotiation successful
0809 15:39:43.229236 (+ 71us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":56}
I20260809 15:39:43.230417 23871 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:39:43.222151 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.89.254:40071 (local address 127.3.89.194:48809)
0809 15:39:43.222756 (+ 605us) negotiation.cc:107] Waiting for socket to connect
0809 15:39:43.222794 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0809 15:39:43.224088 (+ 1294us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 15:39:43.224592 (+ 504us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 15:39:43.224598 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 15:39:43.224677 (+ 79us) client_negotiation.cc:189] Negotiated authn=SASL
0809 15:39:43.225210 (+ 533us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:39:43.225223 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:39:43.226223 (+ 1000us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:39:43.226227 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:39:43.227213 (+ 986us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:39:43.227223 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:39:43.227852 (+ 629us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:39:43.227855 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:39:43.227999 (+ 144us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:39:43.228622 (+ 623us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0809 15:39:43.228623 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0809 15:39:43.228660 (+ 37us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0809 15:39:43.228717 (+ 57us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 15:39:43.229031 (+ 314us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 15:39:43.229038 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 15:39:43.229042 (+ 4us) client_negotiation.cc:769] Sending connection context
0809 15:39:43.229130 (+ 88us) client_negotiation.cc:240] Negotiation successful
0809 15:39:43.229419 (+ 289us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":254,"thread_start_us":115,"threads_started":1}
I20260809 15:39:43.231634 23869 heartbeater.cc:347] Connected to a master server at 127.3.89.254:40071
I20260809 15:39:43.231854 23869 heartbeater.cc:464] Registering TS with master...
I20260809 15:39:43.232331 23869 heartbeater.cc:511] Master 127.3.89.254:40071 requested a full tablet report, sending...
I20260809 15:39:43.233364 23513 ts_manager.cc:194] Registered new tserver with Master: 8d1218ae5ef5440a89f06ac7d82572a4 (127.3.89.194:42561)
I20260809 15:39:43.233924 23513 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.3.89.194:48809
W20260809 15:39:43.339287 23873 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 15:39:43.339550 23873 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 15:39:43.339614 23873 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 15:39:43.343595 23873 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 15:39:43.343665 23873 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 15:39:43.343747 23873 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.3.89.195
I20260809 15:39:43.348661 23873 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.3.89.212:39977
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786289732823074-3431-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786289732823074-3431-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.3.89.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786289732823074-3431-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.3.89.195
--webserver_port=0
--tserver_master_addrs=127.3.89.254:40071
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.23873
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.3.89.195
--log_dir=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786289732823074-3431-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 09 Aug 2026 14:46:55 UTC on ccycloud.quasar-yswkmf.root.comops.site
build id 4576540
I20260809 15:39:43.349653 23873 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 15:39:43.351028 23873 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 15:39:43.360442 23878 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 15:39:43.360433 23879 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 15:39:43.360433 23881 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 15:39:43.360841 23873 server_base.cc:1034] running on GCE node
I20260809 15:39:43.361220 23873 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 15:39:43.361809 23873 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 15:39:43.363037 23873 hybrid_clock.cc:648] HybridClock initialized: now 1786289983363012 us; error 46 us; skew 500 ppm
I20260809 15:39:43.364698 23873 webserver.cc:460] Webserver started at http://127.3.89.195:42627/ using document root <none> and password file <none>
I20260809 15:39:43.365267 23873 fs_manager.cc:359] Metadata directory not provided
I20260809 15:39:43.365362 23873 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 15:39:43.365583 23873 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 15:39:43.367581 23873 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskm314YR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786289732823074-3431-0/minicluster-data/ts-2/data/instance:
uuid: "16715ec3bb96485eb611757a55763228"
format_stamp: "Formatted at 2026-08-09 15:39:43 on dist-test-slave-2w3w"
server_key: "ed4d484b5e1827eda9aa8f88e7fb40df"
server_key_iv: "62d9e05ddab3ab95217d7e73b6c5a548"
server_key_version: "encryptionkey@0"
I20260809 15:39:43.368157 23873 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskm314YR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786289732823074-3431-0/minicluster-data/ts-2/wal/instance:
uuid: "16715ec3bb96485eb611757a55763228"
format_stamp: "Formatted at 2026-08-09 15:39:43 on dist-test-slave-2w3w"
server_key: "ed4d484b5e1827eda9aa8f88e7fb40df"
server_key_iv: "62d9e05ddab3ab95217d7e73b6c5a548"
server_key_version: "encryptionkey@0"
I20260809 15:39:43.371574 23873 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.003s sys 0.000s
I20260809 15:39:43.373828 23887 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 15:39:43.375030 23873 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260809 15:39:43.375152 23873 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskm314YR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786289732823074-3431-0/minicluster-data/ts-2/data,/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786289732823074-3431-0/minicluster-data/ts-2/wal
uuid: "16715ec3bb96485eb611757a55763228"
format_stamp: "Formatted at 2026-08-09 15:39:43 on dist-test-slave-2w3w"
server_key: "ed4d484b5e1827eda9aa8f88e7fb40df"
server_key_iv: "62d9e05ddab3ab95217d7e73b6c5a548"
server_key_version: "encryptionkey@0"
I20260809 15:39:43.375258 23873 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskm314YR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786289732823074-3431-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskm314YR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786289732823074-3431-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskm314YR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786289732823074-3431-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 15:39:43.387756 23873 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 15:39:43.388358 23873 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 15:39:43.388545 23873 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 15:39:43.389091 23873 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 15:39:43.389981 23873 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 15:39:43.390053 23873 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 15:39:43.390126 23873 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 15:39:43.390169 23873 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 15:39:43.398778 23873 rpc_server.cc:307] RPC server started. Bound to: 127.3.89.195:46653
I20260809 15:39:43.398851 24013 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.3.89.195:46653 every 8 connection(s)
I20260809 15:39:43.399690 23873 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskm314YR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786289732823074-3431-0/minicluster-data/ts-2/data/info.pb
I20260809 15:39:43.405303 3431 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskm314YR/build/debug/bin/kudu as pid 23873
I20260809 15:39:43.405411 3431 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskm314YR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786289732823074-3431-0/minicluster-data/ts-2/wal/instance
I20260809 15:39:43.405751 3431 external_mini_cluster.cc:1428] Setting key c767626174320dc78380a5a2cdd16af5
I20260809 15:39:43.408900 23555 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:39:43.401411 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.89.195:52443 (local address 127.3.89.254:40071)
0809 15:39:43.401563 (+ 152us) server_negotiation.cc:206] Beginning negotiation
0809 15:39:43.401567 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 15:39:43.402450 (+ 883us) server_negotiation.cc:407] Connection header received
0809 15:39:43.403865 (+ 1415us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 15:39:43.403868 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 15:39:43.403916 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 15:39:43.403989 (+ 73us) server_negotiation.cc:226] Negotiated authn=SASL
0809 15:39:43.404757 (+ 768us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:39:43.405508 (+ 751us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:39:43.407066 (+ 1558us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:39:43.407353 (+ 287us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:39:43.407471 (+ 118us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:39:43.408555 (+ 1084us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 15:39:43.408558 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 15:39:43.408560 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 15:39:43.408571 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0809 15:39:43.408589 (+ 18us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 15:39:43.408597 (+ 8us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 15:39:43.408662 (+ 65us) server_negotiation.cc:1035] Waiting for connection context
0809 15:39:43.408746 (+ 84us) server_negotiation.cc:299] Negotiation successful
0809 15:39:43.408795 (+ 49us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":63}
I20260809 15:39:43.409724 24016 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:39:43.401707 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.89.254:40071 (local address 127.3.89.195:52443)
0809 15:39:43.402321 (+ 614us) negotiation.cc:107] Waiting for socket to connect
0809 15:39:43.402360 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0809 15:39:43.403589 (+ 1229us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 15:39:43.404113 (+ 524us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 15:39:43.404119 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 15:39:43.404194 (+ 75us) client_negotiation.cc:189] Negotiated authn=SASL
0809 15:39:43.404636 (+ 442us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:39:43.404647 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:39:43.405645 (+ 998us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:39:43.405648 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:39:43.406906 (+ 1258us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:39:43.406917 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:39:43.407498 (+ 581us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:39:43.407501 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:39:43.407640 (+ 139us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:39:43.408320 (+ 680us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0809 15:39:43.408321 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0809 15:39:43.408359 (+ 38us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0809 15:39:43.408417 (+ 58us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 15:39:43.408660 (+ 243us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 15:39:43.408664 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 15:39:43.408668 (+ 4us) client_negotiation.cc:769] Sending connection context
0809 15:39:43.408748 (+ 80us) client_negotiation.cc:240] Negotiation successful
0809 15:39:43.408958 (+ 210us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":263,"thread_start_us":114,"threads_started":1}
I20260809 15:39:43.410816 24014 heartbeater.cc:347] Connected to a master server at 127.3.89.254:40071
I20260809 15:39:43.411048 24014 heartbeater.cc:464] Registering TS with master...
I20260809 15:39:43.411497 24014 heartbeater.cc:511] Master 127.3.89.254:40071 requested a full tablet report, sending...
I20260809 15:39:43.412385 23513 ts_manager.cc:194] Registered new tserver with Master: 16715ec3bb96485eb611757a55763228 (127.3.89.195:46653)
I20260809 15:39:43.412941 23513 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.3.89.195:52443
I20260809 15:39:43.419968 3431 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260809 15:39:43.425885 23555 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:39:43.421898 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50774 (local address 127.3.89.254:40071)
0809 15:39:43.422022 (+ 124us) server_negotiation.cc:206] Beginning negotiation
0809 15:39:43.422029 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 15:39:43.422217 (+ 188us) server_negotiation.cc:407] Connection header received
0809 15:39:43.422387 (+ 170us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 15:39:43.422390 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 15:39:43.422432 (+ 42us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 15:39:43.422515 (+ 83us) server_negotiation.cc:226] Negotiated authn=SASL
0809 15:39:43.422791 (+ 276us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:39:43.423395 (+ 604us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:39:43.424245 (+ 850us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:39:43.424474 (+ 229us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:39:43.424584 (+ 110us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:39:43.425373 (+ 789us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 15:39:43.425377 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 15:39:43.425379 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 15:39:43.425392 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0809 15:39:43.425413 (+ 21us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 15:39:43.425427 (+ 14us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 15:39:43.425555 (+ 128us) server_negotiation.cc:1035] Waiting for connection context
0809 15:39:43.425703 (+ 148us) server_negotiation.cc:299] Negotiation successful
0809 15:39:43.425742 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":48}
I20260809 15:39:43.428452 23513 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:50774:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260809 15:39:43.431517 23513 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260809 15:39:43.445581 24027 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:39:43.439723 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.89.193:41675 (local address 127.0.0.1:38022)
0809 15:39:43.440392 (+ 669us) negotiation.cc:107] Waiting for socket to connect
0809 15:39:43.440443 (+ 51us) client_negotiation.cc:174] Beginning negotiation
0809 15:39:43.440583 (+ 140us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 15:39:43.441355 (+ 772us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 15:39:43.441361 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 15:39:43.441383 (+ 22us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 15:39:43.441514 (+ 131us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:39:43.441523 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:39:43.442818 (+ 1295us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:39:43.442821 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:39:43.444227 (+ 1406us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:39:43.444237 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:39:43.445054 (+ 817us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:39:43.445057 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:39:43.445164 (+ 107us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:39:43.445202 (+ 38us) client_negotiation.cc:769] Sending connection context
0809 15:39:43.445316 (+ 114us) client_negotiation.cc:240] Negotiation successful
0809 15:39:43.445436 (+ 120us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":517,"thread_start_us":96,"threads_started":1}
I20260809 15:39:43.445606 24029 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:39:43.439927 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38022 (local address 127.3.89.193:41675)
0809 15:39:43.440914 (+ 987us) server_negotiation.cc:206] Beginning negotiation
0809 15:39:43.440926 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0809 15:39:43.440951 (+ 25us) server_negotiation.cc:407] Connection header received
0809 15:39:43.441029 (+ 78us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 15:39:43.441042 (+ 13us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 15:39:43.441198 (+ 156us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 15:39:43.441364 (+ 166us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 15:39:43.441639 (+ 275us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:39:43.442691 (+ 1052us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:39:43.444368 (+ 1677us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:39:43.444919 (+ 551us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:39:43.445048 (+ 129us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:39:43.445238 (+ 190us) server_negotiation.cc:1035] Waiting for connection context
0809 15:39:43.445357 (+ 119us) server_negotiation.cc:299] Negotiation successful
0809 15:39:43.445446 (+ 89us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":868,"thread_start_us":133,"threads_started":1}
I20260809 15:39:43.446152 24028 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:39:43.439723 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39736 (local address 127.3.89.194:42561)
0809 15:39:43.440520 (+ 797us) server_negotiation.cc:206] Beginning negotiation
0809 15:39:43.440533 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0809 15:39:43.441333 (+ 800us) server_negotiation.cc:407] Connection header received
0809 15:39:43.441545 (+ 212us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 15:39:43.441549 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 15:39:43.441713 (+ 164us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 15:39:43.441835 (+ 122us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 15:39:43.442235 (+ 400us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:39:43.443120 (+ 885us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:39:43.444549 (+ 1429us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:39:43.445291 (+ 742us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:39:43.445383 (+ 92us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:39:43.445540 (+ 157us) server_negotiation.cc:1035] Waiting for connection context
0809 15:39:43.445885 (+ 345us) server_negotiation.cc:299] Negotiation successful
0809 15:39:43.445979 (+ 94us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":673,"thread_start_us":156,"threads_started":1}
I20260809 15:39:43.446239 24030 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:39:43.440454 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.89.195:46653 (local address 127.0.0.1:48600)
0809 15:39:43.440709 (+ 255us) negotiation.cc:107] Waiting for socket to connect
0809 15:39:43.440724 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0809 15:39:43.440808 (+ 84us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 15:39:43.441832 (+ 1024us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 15:39:43.441834 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 15:39:43.441850 (+ 16us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 15:39:43.441949 (+ 99us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:39:43.441959 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:39:43.443299 (+ 1340us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:39:43.443303 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:39:43.444277 (+ 974us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:39:43.444287 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:39:43.445609 (+ 1322us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:39:43.445613 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:39:43.445935 (+ 322us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:39:43.445956 (+ 21us) client_negotiation.cc:769] Sending connection context
0809 15:39:43.446031 (+ 75us) client_negotiation.cc:240] Negotiation successful
0809 15:39:43.446102 (+ 71us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":188,"mutex_wait_us":57,"thread_start_us":109,"threads_started":1}
I20260809 15:39:43.446424 24031 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:39:43.440881 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48600 (local address 127.3.89.195:46653)
0809 15:39:43.441222 (+ 341us) server_negotiation.cc:206] Beginning negotiation
0809 15:39:43.441235 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0809 15:39:43.441260 (+ 25us) server_negotiation.cc:407] Connection header received
0809 15:39:43.441335 (+ 75us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 15:39:43.441340 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 15:39:43.441523 (+ 183us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 15:39:43.441658 (+ 135us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 15:39:43.442091 (+ 433us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:39:43.443159 (+ 1068us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:39:43.444430 (+ 1271us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:39:43.445208 (+ 778us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:39:43.445362 (+ 154us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:39:43.445540 (+ 178us) server_negotiation.cc:1035] Waiting for connection context
0809 15:39:43.446088 (+ 548us) server_negotiation.cc:299] Negotiation successful
0809 15:39:43.446188 (+ 100us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":246,"thread_start_us":155,"threads_started":1}
I20260809 15:39:43.446563 24026 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:39:43.439603 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.89.194:42561 (local address 127.0.0.1:39736)
0809 15:39:43.441213 (+ 1610us) negotiation.cc:107] Waiting for socket to connect
0809 15:39:43.441231 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0809 15:39:43.441403 (+ 172us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 15:39:43.442005 (+ 602us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 15:39:43.442009 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 15:39:43.442030 (+ 21us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 15:39:43.442115 (+ 85us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:39:43.442119 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:39:43.443255 (+ 1136us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:39:43.443258 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:39:43.444420 (+ 1162us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:39:43.444429 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:39:43.445609 (+ 1180us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:39:43.445613 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:39:43.445720 (+ 107us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:39:43.445742 (+ 22us) client_negotiation.cc:769] Sending connection context
0809 15:39:43.446326 (+ 584us) client_negotiation.cc:240] Negotiation successful
0809 15:39:43.446411 (+ 85us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":606,"mutex_wait_us":28,"thread_start_us":90,"threads_started":1}
I20260809 15:39:43.448577 23649 tablet_service.cc:1467] Processing CreateTablet for tablet cfcbae6562cb410bb7e113a6be4ee6fc (DEFAULT_TABLE table=test-table [id=474268a7242f4de4954d2050518ff7bf]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 15:39:43.449007 23939 tablet_service.cc:1467] Processing CreateTablet for tablet cfcbae6562cb410bb7e113a6be4ee6fc (DEFAULT_TABLE table=test-table [id=474268a7242f4de4954d2050518ff7bf]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 15:39:43.449007 23791 tablet_service.cc:1467] Processing CreateTablet for tablet cfcbae6562cb410bb7e113a6be4ee6fc (DEFAULT_TABLE table=test-table [id=474268a7242f4de4954d2050518ff7bf]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 15:39:43.449417 23649 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet cfcbae6562cb410bb7e113a6be4ee6fc. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 15:39:43.449934 23939 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet cfcbae6562cb410bb7e113a6be4ee6fc. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 15:39:43.449934 23791 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet cfcbae6562cb410bb7e113a6be4ee6fc. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 15:39:43.455809 24032 tablet_bootstrap.cc:492] T cfcbae6562cb410bb7e113a6be4ee6fc P 8460063038f14ab5898b25822335d44a: Bootstrap starting.
I20260809 15:39:43.457063 24033 tablet_bootstrap.cc:492] T cfcbae6562cb410bb7e113a6be4ee6fc P 16715ec3bb96485eb611757a55763228: Bootstrap starting.
I20260809 15:39:43.459276 24034 tablet_bootstrap.cc:492] T cfcbae6562cb410bb7e113a6be4ee6fc P 8d1218ae5ef5440a89f06ac7d82572a4: Bootstrap starting.
I20260809 15:39:43.459408 24032 tablet_bootstrap.cc:654] T cfcbae6562cb410bb7e113a6be4ee6fc P 8460063038f14ab5898b25822335d44a: Neither blocks nor log segments found. Creating new log.
I20260809 15:39:43.459408 24033 tablet_bootstrap.cc:654] T cfcbae6562cb410bb7e113a6be4ee6fc P 16715ec3bb96485eb611757a55763228: Neither blocks nor log segments found. Creating new log.
I20260809 15:39:43.460295 24032 log.cc:824] T cfcbae6562cb410bb7e113a6be4ee6fc P 8460063038f14ab5898b25822335d44a: Log is configured to *not* fsync() on all Append() calls
I20260809 15:39:43.460295 24033 log.cc:824] T cfcbae6562cb410bb7e113a6be4ee6fc P 16715ec3bb96485eb611757a55763228: Log is configured to *not* fsync() on all Append() calls
I20260809 15:39:43.461102 24034 tablet_bootstrap.cc:654] T cfcbae6562cb410bb7e113a6be4ee6fc P 8d1218ae5ef5440a89f06ac7d82572a4: Neither blocks nor log segments found. Creating new log.
I20260809 15:39:43.461758 24034 log.cc:824] T cfcbae6562cb410bb7e113a6be4ee6fc P 8d1218ae5ef5440a89f06ac7d82572a4: Log is configured to *not* fsync() on all Append() calls
I20260809 15:39:43.462301 24033 tablet_bootstrap.cc:492] T cfcbae6562cb410bb7e113a6be4ee6fc P 16715ec3bb96485eb611757a55763228: No bootstrap required, opened a new log
I20260809 15:39:43.462301 24032 tablet_bootstrap.cc:492] T cfcbae6562cb410bb7e113a6be4ee6fc P 8460063038f14ab5898b25822335d44a: No bootstrap required, opened a new log
I20260809 15:39:43.462512 24032 ts_tablet_manager.cc:1397] T cfcbae6562cb410bb7e113a6be4ee6fc P 8460063038f14ab5898b25822335d44a: Time spent bootstrapping tablet: real 0.007s user 0.004s sys 0.000s
I20260809 15:39:43.462654 24033 ts_tablet_manager.cc:1397] T cfcbae6562cb410bb7e113a6be4ee6fc P 16715ec3bb96485eb611757a55763228: Time spent bootstrapping tablet: real 0.006s user 0.005s sys 0.000s
I20260809 15:39:43.463793 24034 tablet_bootstrap.cc:492] T cfcbae6562cb410bb7e113a6be4ee6fc P 8d1218ae5ef5440a89f06ac7d82572a4: No bootstrap required, opened a new log
I20260809 15:39:43.463990 24034 ts_tablet_manager.cc:1397] T cfcbae6562cb410bb7e113a6be4ee6fc P 8d1218ae5ef5440a89f06ac7d82572a4: Time spent bootstrapping tablet: real 0.005s user 0.003s sys 0.000s
I20260809 15:39:43.466854 24034 raft_consensus.cc:348] T cfcbae6562cb410bb7e113a6be4ee6fc P 8d1218ae5ef5440a89f06ac7d82572a4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8d1218ae5ef5440a89f06ac7d82572a4" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42561 } } peers { permanent_uuid: "8460063038f14ab5898b25822335d44a" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 41675 } } peers { permanent_uuid: "16715ec3bb96485eb611757a55763228" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 46653 } }
I20260809 15:39:43.466908 24032 raft_consensus.cc:348] T cfcbae6562cb410bb7e113a6be4ee6fc P 8460063038f14ab5898b25822335d44a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8d1218ae5ef5440a89f06ac7d82572a4" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42561 } } peers { permanent_uuid: "8460063038f14ab5898b25822335d44a" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 41675 } } peers { permanent_uuid: "16715ec3bb96485eb611757a55763228" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 46653 } }
I20260809 15:39:43.466908 24033 raft_consensus.cc:348] T cfcbae6562cb410bb7e113a6be4ee6fc P 16715ec3bb96485eb611757a55763228 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8d1218ae5ef5440a89f06ac7d82572a4" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42561 } } peers { permanent_uuid: "8460063038f14ab5898b25822335d44a" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 41675 } } peers { permanent_uuid: "16715ec3bb96485eb611757a55763228" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 46653 } }
I20260809 15:39:43.467154 24034 raft_consensus.cc:374] T cfcbae6562cb410bb7e113a6be4ee6fc P 8d1218ae5ef5440a89f06ac7d82572a4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 15:39:43.467175 24032 raft_consensus.cc:374] T cfcbae6562cb410bb7e113a6be4ee6fc P 8460063038f14ab5898b25822335d44a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 15:39:43.467175 24033 raft_consensus.cc:374] T cfcbae6562cb410bb7e113a6be4ee6fc P 16715ec3bb96485eb611757a55763228 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 15:39:43.467211 24034 raft_consensus.cc:729] T cfcbae6562cb410bb7e113a6be4ee6fc P 8d1218ae5ef5440a89f06ac7d82572a4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8d1218ae5ef5440a89f06ac7d82572a4, State: Initialized, Role: FOLLOWER
I20260809 15:39:43.467236 24032 raft_consensus.cc:729] T cfcbae6562cb410bb7e113a6be4ee6fc P 8460063038f14ab5898b25822335d44a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8460063038f14ab5898b25822335d44a, State: Initialized, Role: FOLLOWER
I20260809 15:39:43.467236 24033 raft_consensus.cc:729] T cfcbae6562cb410bb7e113a6be4ee6fc P 16715ec3bb96485eb611757a55763228 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 16715ec3bb96485eb611757a55763228, State: Initialized, Role: FOLLOWER
I20260809 15:39:43.467648 24034 consensus_queue.cc:260] T cfcbae6562cb410bb7e113a6be4ee6fc P 8d1218ae5ef5440a89f06ac7d82572a4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8d1218ae5ef5440a89f06ac7d82572a4" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42561 } } peers { permanent_uuid: "8460063038f14ab5898b25822335d44a" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 41675 } } peers { permanent_uuid: "16715ec3bb96485eb611757a55763228" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 46653 } }
I20260809 15:39:43.467694 24032 consensus_queue.cc:260] T cfcbae6562cb410bb7e113a6be4ee6fc P 8460063038f14ab5898b25822335d44a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8d1218ae5ef5440a89f06ac7d82572a4" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42561 } } peers { permanent_uuid: "8460063038f14ab5898b25822335d44a" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 41675 } } peers { permanent_uuid: "16715ec3bb96485eb611757a55763228" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 46653 } }
I20260809 15:39:43.467694 24033 consensus_queue.cc:260] T cfcbae6562cb410bb7e113a6be4ee6fc P 16715ec3bb96485eb611757a55763228 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8d1218ae5ef5440a89f06ac7d82572a4" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42561 } } peers { permanent_uuid: "8460063038f14ab5898b25822335d44a" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 41675 } } peers { permanent_uuid: "16715ec3bb96485eb611757a55763228" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 46653 } }
I20260809 15:39:43.468266 23869 heartbeater.cc:503] Master 127.3.89.254:40071 was elected leader, sending a full tablet report...
I20260809 15:39:43.468596 24034 ts_tablet_manager.cc:1428] T cfcbae6562cb410bb7e113a6be4ee6fc P 8d1218ae5ef5440a89f06ac7d82572a4: Time spent starting tablet: real 0.005s user 0.005s sys 0.000s
I20260809 15:39:43.468631 24032 ts_tablet_manager.cc:1428] T cfcbae6562cb410bb7e113a6be4ee6fc P 8460063038f14ab5898b25822335d44a: Time spent starting tablet: real 0.006s user 0.007s sys 0.000s
I20260809 15:39:43.468957 23722 heartbeater.cc:503] Master 127.3.89.254:40071 was elected leader, sending a full tablet report...
I20260809 15:39:43.469539 24033 ts_tablet_manager.cc:1428] T cfcbae6562cb410bb7e113a6be4ee6fc P 16715ec3bb96485eb611757a55763228: Time spent starting tablet: real 0.007s user 0.002s sys 0.003s
I20260809 15:39:43.469879 24014 heartbeater.cc:503] Master 127.3.89.254:40071 was elected leader, sending a full tablet report...
W20260809 15:39:43.471427 23870 tablet.cc:2369] T cfcbae6562cb410bb7e113a6be4ee6fc P 8d1218ae5ef5440a89f06ac7d82572a4: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260809 15:39:43.539398 23723 tablet.cc:2369] T cfcbae6562cb410bb7e113a6be4ee6fc P 8460063038f14ab5898b25822335d44a: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260809 15:39:43.650758 24015 tablet.cc:2369] T cfcbae6562cb410bb7e113a6be4ee6fc P 16715ec3bb96485eb611757a55763228: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260809 15:39:43.673759 24041 raft_consensus.cc:482] T cfcbae6562cb410bb7e113a6be4ee6fc P 16715ec3bb96485eb611757a55763228 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 15:39:43.674000 24041 raft_consensus.cc:504] T cfcbae6562cb410bb7e113a6be4ee6fc P 16715ec3bb96485eb611757a55763228 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8d1218ae5ef5440a89f06ac7d82572a4" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42561 } } peers { permanent_uuid: "8460063038f14ab5898b25822335d44a" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 41675 } } peers { permanent_uuid: "16715ec3bb96485eb611757a55763228" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 46653 } }
I20260809 15:39:43.675202 24041 leader_election.cc:288] T cfcbae6562cb410bb7e113a6be4ee6fc P 16715ec3bb96485eb611757a55763228 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 8d1218ae5ef5440a89f06ac7d82572a4 (127.3.89.194:42561), 8460063038f14ab5898b25822335d44a (127.3.89.193:41675)
I20260809 15:39:43.680087 24028 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:39:43.675713 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.89.195:54437 (local address 127.3.89.194:42561)
0809 15:39:43.675854 (+ 141us) server_negotiation.cc:206] Beginning negotiation
0809 15:39:43.675858 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 15:39:43.675874 (+ 16us) server_negotiation.cc:407] Connection header received
0809 15:39:43.675936 (+ 62us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 15:39:43.675938 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 15:39:43.675980 (+ 42us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 15:39:43.676055 (+ 75us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 15:39:43.676384 (+ 329us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:39:43.677194 (+ 810us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:39:43.678705 (+ 1511us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:39:43.679372 (+ 667us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:39:43.679508 (+ 136us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:39:43.679551 (+ 43us) server_negotiation.cc:1035] Waiting for connection context
0809 15:39:43.679885 (+ 334us) server_negotiation.cc:299] Negotiation successful
0809 15:39:43.679956 (+ 71us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":54}
I20260809 15:39:43.680087 24029 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:39:43.675626 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.89.195:33755 (local address 127.3.89.193:41675)
0809 15:39:43.675767 (+ 141us) server_negotiation.cc:206] Beginning negotiation
0809 15:39:43.675772 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 15:39:43.676121 (+ 349us) server_negotiation.cc:407] Connection header received
0809 15:39:43.676248 (+ 127us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 15:39:43.676251 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 15:39:43.676295 (+ 44us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 15:39:43.676396 (+ 101us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 15:39:43.676633 (+ 237us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:39:43.677346 (+ 713us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:39:43.678705 (+ 1359us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:39:43.679383 (+ 678us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:39:43.679502 (+ 119us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:39:43.679551 (+ 49us) server_negotiation.cc:1035] Waiting for connection context
0809 15:39:43.679885 (+ 334us) server_negotiation.cc:299] Negotiation successful
0809 15:39:43.679956 (+ 71us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":57}
I20260809 15:39:43.680116 24016 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:39:43.675494 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.89.194:42561 (local address 127.3.89.195:54437)
0809 15:39:43.675685 (+ 191us) negotiation.cc:107] Waiting for socket to connect
0809 15:39:43.675704 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0809 15:39:43.675803 (+ 99us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 15:39:43.676087 (+ 284us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 15:39:43.676091 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 15:39:43.676131 (+ 40us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 15:39:43.676264 (+ 133us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:39:43.676277 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:39:43.677301 (+ 1024us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:39:43.677303 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:39:43.678569 (+ 1266us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:39:43.678580 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:39:43.679502 (+ 922us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:39:43.679506 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:39:43.679718 (+ 212us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:39:43.679776 (+ 58us) client_negotiation.cc:769] Sending connection context
0809 15:39:43.679853 (+ 77us) client_negotiation.cc:240] Negotiation successful
0809 15:39:43.679936 (+ 83us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":87,"mutex_wait_us":113,"spinlock_wait_cycles":27648}
I20260809 15:39:43.680125 24043 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:39:43.675494 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.89.193:41675 (local address 127.3.89.195:33755)
0809 15:39:43.675993 (+ 499us) negotiation.cc:107] Waiting for socket to connect
0809 15:39:43.676004 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0809 15:39:43.676136 (+ 132us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 15:39:43.676413 (+ 277us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 15:39:43.676415 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 15:39:43.676430 (+ 15us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 15:39:43.676520 (+ 90us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:39:43.676524 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:39:43.677436 (+ 912us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:39:43.677439 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:39:43.678573 (+ 1134us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:39:43.678580 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:39:43.679502 (+ 922us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:39:43.679506 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:39:43.679701 (+ 195us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:39:43.679767 (+ 66us) client_negotiation.cc:769] Sending connection context
0809 15:39:43.679853 (+ 86us) client_negotiation.cc:240] Negotiation successful
0809 15:39:43.679935 (+ 82us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":433,"mutex_wait_us":226,"spinlock_wait_cycles":7808,"thread_start_us":116,"threads_started":1}
I20260809 15:39:43.680851 23815 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "cfcbae6562cb410bb7e113a6be4ee6fc" candidate_uuid: "16715ec3bb96485eb611757a55763228" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8d1218ae5ef5440a89f06ac7d82572a4" is_pre_election: true
I20260809 15:39:43.680991 23671 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "cfcbae6562cb410bb7e113a6be4ee6fc" candidate_uuid: "16715ec3bb96485eb611757a55763228" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8460063038f14ab5898b25822335d44a" is_pre_election: true
I20260809 15:39:43.681175 23815 raft_consensus.cc:2454] T cfcbae6562cb410bb7e113a6be4ee6fc P 8d1218ae5ef5440a89f06ac7d82572a4 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 16715ec3bb96485eb611757a55763228 in term 0.
I20260809 15:39:43.681311 23671 raft_consensus.cc:2454] T cfcbae6562cb410bb7e113a6be4ee6fc P 8460063038f14ab5898b25822335d44a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 16715ec3bb96485eb611757a55763228 in term 0.
I20260809 15:39:43.681731 23888 leader_election.cc:302] T cfcbae6562cb410bb7e113a6be4ee6fc P 16715ec3bb96485eb611757a55763228 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 16715ec3bb96485eb611757a55763228, 8d1218ae5ef5440a89f06ac7d82572a4; no voters:
I20260809 15:39:43.682037 24041 raft_consensus.cc:2781] T cfcbae6562cb410bb7e113a6be4ee6fc P 16715ec3bb96485eb611757a55763228 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260809 15:39:43.682138 24041 raft_consensus.cc:482] T cfcbae6562cb410bb7e113a6be4ee6fc P 16715ec3bb96485eb611757a55763228 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 15:39:43.682199 24041 raft_consensus.cc:3037] T cfcbae6562cb410bb7e113a6be4ee6fc P 16715ec3bb96485eb611757a55763228 [term 0 FOLLOWER]: Advancing to term 1
I20260809 15:39:43.683331 24041 raft_consensus.cc:504] T cfcbae6562cb410bb7e113a6be4ee6fc P 16715ec3bb96485eb611757a55763228 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8d1218ae5ef5440a89f06ac7d82572a4" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42561 } } peers { permanent_uuid: "8460063038f14ab5898b25822335d44a" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 41675 } } peers { permanent_uuid: "16715ec3bb96485eb611757a55763228" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 46653 } }
I20260809 15:39:43.683712 24041 leader_election.cc:288] T cfcbae6562cb410bb7e113a6be4ee6fc P 16715ec3bb96485eb611757a55763228 [CANDIDATE]: Term 1 election: Requested vote from peers 8d1218ae5ef5440a89f06ac7d82572a4 (127.3.89.194:42561), 8460063038f14ab5898b25822335d44a (127.3.89.193:41675)
I20260809 15:39:43.684065 23671 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "cfcbae6562cb410bb7e113a6be4ee6fc" candidate_uuid: "16715ec3bb96485eb611757a55763228" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8460063038f14ab5898b25822335d44a"
I20260809 15:39:43.684108 23815 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "cfcbae6562cb410bb7e113a6be4ee6fc" candidate_uuid: "16715ec3bb96485eb611757a55763228" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8d1218ae5ef5440a89f06ac7d82572a4"
I20260809 15:39:43.684203 23671 raft_consensus.cc:3037] T cfcbae6562cb410bb7e113a6be4ee6fc P 8460063038f14ab5898b25822335d44a [term 0 FOLLOWER]: Advancing to term 1
I20260809 15:39:43.684237 23815 raft_consensus.cc:3037] T cfcbae6562cb410bb7e113a6be4ee6fc P 8d1218ae5ef5440a89f06ac7d82572a4 [term 0 FOLLOWER]: Advancing to term 1
I20260809 15:39:43.685221 23671 raft_consensus.cc:2454] T cfcbae6562cb410bb7e113a6be4ee6fc P 8460063038f14ab5898b25822335d44a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 16715ec3bb96485eb611757a55763228 in term 1.
I20260809 15:39:43.685299 23815 raft_consensus.cc:2454] T cfcbae6562cb410bb7e113a6be4ee6fc P 8d1218ae5ef5440a89f06ac7d82572a4 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 16715ec3bb96485eb611757a55763228 in term 1.
I20260809 15:39:43.685582 23891 leader_election.cc:302] T cfcbae6562cb410bb7e113a6be4ee6fc P 16715ec3bb96485eb611757a55763228 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 16715ec3bb96485eb611757a55763228, 8460063038f14ab5898b25822335d44a; no voters:
I20260809 15:39:43.685766 24041 raft_consensus.cc:2781] T cfcbae6562cb410bb7e113a6be4ee6fc P 16715ec3bb96485eb611757a55763228 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 15:39:43.686012 24041 raft_consensus.cc:686] T cfcbae6562cb410bb7e113a6be4ee6fc P 16715ec3bb96485eb611757a55763228 [term 1 LEADER]: Becoming Leader. State: Replica: 16715ec3bb96485eb611757a55763228, State: Running, Role: LEADER
I20260809 15:39:43.686347 24041 consensus_queue.cc:237] T cfcbae6562cb410bb7e113a6be4ee6fc P 16715ec3bb96485eb611757a55763228 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8d1218ae5ef5440a89f06ac7d82572a4" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42561 } } peers { permanent_uuid: "8460063038f14ab5898b25822335d44a" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 41675 } } peers { permanent_uuid: "16715ec3bb96485eb611757a55763228" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 46653 } }
I20260809 15:39:43.689383 23511 catalog_manager.cc:5495] T cfcbae6562cb410bb7e113a6be4ee6fc P 16715ec3bb96485eb611757a55763228 reported cstate change: term changed from 0 to 1, leader changed from <none> to 16715ec3bb96485eb611757a55763228 (127.3.89.195). New cstate: current_term: 1 leader_uuid: "16715ec3bb96485eb611757a55763228" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8d1218ae5ef5440a89f06ac7d82572a4" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42561 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8460063038f14ab5898b25822335d44a" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 41675 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "16715ec3bb96485eb611757a55763228" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 46653 } health_report { overall_health: HEALTHY } } }
I20260809 15:39:43.739807 24031 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:39:43.736208 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48604 (local address 127.3.89.195:46653)
0809 15:39:43.736343 (+ 135us) server_negotiation.cc:206] Beginning negotiation
0809 15:39:43.736347 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 15:39:43.736359 (+ 12us) server_negotiation.cc:407] Connection header received
0809 15:39:43.736427 (+ 68us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 15:39:43.736429 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 15:39:43.736484 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 15:39:43.736600 (+ 116us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 15:39:43.736864 (+ 264us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:39:43.737369 (+ 505us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:39:43.738263 (+ 894us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:39:43.738486 (+ 223us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:39:43.738594 (+ 108us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:39:43.738907 (+ 313us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 15:39:43.739368 (+ 461us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 15:39:43.739533 (+ 165us) server_negotiation.cc:1035] Waiting for connection context
0809 15:39:43.739631 (+ 98us) server_negotiation.cc:299] Negotiation successful
0809 15:39:43.739667 (+ 36us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":54}
I20260809 15:39:43.747612 23815 raft_consensus.cc:1261] T cfcbae6562cb410bb7e113a6be4ee6fc P 8d1218ae5ef5440a89f06ac7d82572a4 [term 1 FOLLOWER]: Refusing update from remote peer 16715ec3bb96485eb611757a55763228: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 15:39:43.747612 23671 raft_consensus.cc:1261] T cfcbae6562cb410bb7e113a6be4ee6fc P 8460063038f14ab5898b25822335d44a [term 1 FOLLOWER]: Refusing update from remote peer 16715ec3bb96485eb611757a55763228: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 15:39:43.748401 24041 consensus_queue.cc:1035] T cfcbae6562cb410bb7e113a6be4ee6fc P 16715ec3bb96485eb611757a55763228 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8d1218ae5ef5440a89f06ac7d82572a4" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 42561 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 15:39:43.748662 24044 consensus_queue.cc:1035] T cfcbae6562cb410bb7e113a6be4ee6fc P 16715ec3bb96485eb611757a55763228 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8460063038f14ab5898b25822335d44a" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 41675 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 15:39:43.755786 24047 mvcc.cc:205] Tried to move back new op lower bound from 7316643773421170688 to 7316643773181837312. Current Snapshot: MvccSnapshot[applied={T|T < 7316643773421170688}]
I20260809 15:39:43.756011 24049 mvcc.cc:205] Tried to move back new op lower bound from 7316643773421170688 to 7316643773181837312. Current Snapshot: MvccSnapshot[applied={T|T < 7316643773421170688}]
I20260809 15:39:43.766782 23513 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:50774:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 15:39:43.766997 23513 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:50774:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 15:39:43.769490 23513 catalog_manager.cc:5782] T 00000000000000000000000000000000 P bf24fedf691d442c8cf055457a36c8dc: Sending DeleteTablet for 3 replicas of tablet cfcbae6562cb410bb7e113a6be4ee6fc
I20260809 15:39:43.770236 23649 tablet_service.cc:1514] Processing DeleteTablet for tablet cfcbae6562cb410bb7e113a6be4ee6fc with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 15:39:43 UTC) from {username='slave'} at 127.0.0.1:38022
I20260809 15:39:43.770233 23791 tablet_service.cc:1514] Processing DeleteTablet for tablet cfcbae6562cb410bb7e113a6be4ee6fc with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 15:39:43 UTC) from {username='slave'} at 127.0.0.1:39736
I20260809 15:39:43.770686 24059 tablet_replica.cc:330] T cfcbae6562cb410bb7e113a6be4ee6fc P 8460063038f14ab5898b25822335d44a: stopping tablet replica
I20260809 15:39:43.770684 23939 tablet_service.cc:1514] Processing DeleteTablet for tablet cfcbae6562cb410bb7e113a6be4ee6fc with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 15:39:43 UTC) from {username='slave'} at 127.0.0.1:48600
I20260809 15:39:43.770918 24059 raft_consensus.cc:2229] T cfcbae6562cb410bb7e113a6be4ee6fc P 8460063038f14ab5898b25822335d44a [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 15:39:43.771214 24060 tablet_replica.cc:330] T cfcbae6562cb410bb7e113a6be4ee6fc P 16715ec3bb96485eb611757a55763228: stopping tablet replica
I20260809 15:39:43.771394 24058 tablet_replica.cc:330] T cfcbae6562cb410bb7e113a6be4ee6fc P 8d1218ae5ef5440a89f06ac7d82572a4: stopping tablet replica
I20260809 15:39:43.771561 24060 raft_consensus.cc:2229] T cfcbae6562cb410bb7e113a6be4ee6fc P 16715ec3bb96485eb611757a55763228 [term 1 LEADER]: Raft consensus shutting down.
I20260809 15:39:43.771756 24058 raft_consensus.cc:2229] T cfcbae6562cb410bb7e113a6be4ee6fc P 8d1218ae5ef5440a89f06ac7d82572a4 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 15:39:43.771759 3431 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskm314YR/build/debug/bin/kudu with pid 23580
I20260809 15:39:43.772094 24058 raft_consensus.cc:2258] T cfcbae6562cb410bb7e113a6be4ee6fc P 8d1218ae5ef5440a89f06ac7d82572a4 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 15:39:43.772096 24060 raft_consensus.cc:2258] T cfcbae6562cb410bb7e113a6be4ee6fc P 16715ec3bb96485eb611757a55763228 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 15:39:43.773967 24060 ts_tablet_manager.cc:1905] T cfcbae6562cb410bb7e113a6be4ee6fc P 16715ec3bb96485eb611757a55763228: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 15:39:43.778880 24058 ts_tablet_manager.cc:1905] T cfcbae6562cb410bb7e113a6be4ee6fc P 8d1218ae5ef5440a89f06ac7d82572a4: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 15:39:43.780639 24060 ts_tablet_manager.cc:1918] T cfcbae6562cb410bb7e113a6be4ee6fc P 16715ec3bb96485eb611757a55763228: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 15:39:43.780737 24060 log.cc:1196] T cfcbae6562cb410bb7e113a6be4ee6fc P 16715ec3bb96485eb611757a55763228: Deleting WAL directory at /tmp/dist-test-taskm314YR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786289732823074-3431-0/minicluster-data/ts-2/wal/wals/cfcbae6562cb410bb7e113a6be4ee6fc
I20260809 15:39:43.781193 24060 ts_tablet_manager.cc:1939] T cfcbae6562cb410bb7e113a6be4ee6fc P 16715ec3bb96485eb611757a55763228: Deleting consensus metadata
I20260809 15:39:43.781715 24058 ts_tablet_manager.cc:1918] T cfcbae6562cb410bb7e113a6be4ee6fc P 8d1218ae5ef5440a89f06ac7d82572a4: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 15:39:43.781800 24058 log.cc:1196] T cfcbae6562cb410bb7e113a6be4ee6fc P 8d1218ae5ef5440a89f06ac7d82572a4: Deleting WAL directory at /tmp/dist-test-taskm314YR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786289732823074-3431-0/minicluster-data/ts-1/wal/wals/cfcbae6562cb410bb7e113a6be4ee6fc
I20260809 15:39:43.782104 24058 ts_tablet_manager.cc:1939] T cfcbae6562cb410bb7e113a6be4ee6fc P 8d1218ae5ef5440a89f06ac7d82572a4: Deleting consensus metadata
I20260809 15:39:43.783063 23494 catalog_manager.cc:4841] TS 8d1218ae5ef5440a89f06ac7d82572a4 (127.3.89.194:42561): tablet cfcbae6562cb410bb7e113a6be4ee6fc (table test-table [id=474268a7242f4de4954d2050518ff7bf]) successfully deleted
I20260809 15:39:43.784843 23496 catalog_manager.cc:4841] TS 16715ec3bb96485eb611757a55763228 (127.3.89.195:46653): tablet cfcbae6562cb410bb7e113a6be4ee6fc (table test-table [id=474268a7242f4de4954d2050518ff7bf]) successfully deleted
W20260809 15:39:43.787830 23498 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.3.89.193:41675 (error 108)
I20260809 15:39:43.788671 3431 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskm314YR/build/debug/bin/kudu with pid 23725
I20260809 15:39:43.788884 24026 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:39:43.788539 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.89.193:41675 (local address 127.0.0.1:38032)
0809 15:39:43.788721 (+ 182us) negotiation.cc:107] Waiting for socket to connect
0809 15:39:43.788792 (+ 71us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.89.193:41675: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":96}
W20260809 15:39:43.789137 23498 catalog_manager.cc:4584] TS 8460063038f14ab5898b25822335d44a (127.3.89.193:41675): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet cfcbae6562cb410bb7e113a6be4ee6fc: Network error: Client connection negotiation failed: client connection to 127.3.89.193:41675: connect: Connection refused (error 111)
I20260809 15:39:43.797079 3431 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskm314YR/build/debug/bin/kudu with pid 23873
I20260809 15:39:43.802954 3431 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskm314YR/build/debug/bin/kudu with pid 23476
2026-08-09T15:39:43Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/3 (1229 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/4
Loading random data
Initializing database '/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786289732823074-3431-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 15:39:43 dist-test-slave-2w3w krb5kdc[24064](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 15:39:43 dist-test-slave-2w3w krb5kdc[24064](info): set up 2 sockets
Aug 09 15:39:43 dist-test-slave-2w3w krb5kdc[24064](info): commencing operation
krb5kdc: starting...
W20260809 15:39:44.892720 3431 mini_kdc.cc:121] Time spent starting KDC: real 1.067s user 0.000s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786289732823074-3431-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786289732823074-3431-0/krb5kdc/test-user.keytab.
Aug 09 15:39:44 dist-test-slave-2w3w krb5kdc[24064](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786289984, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T15:39:44Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T15:39:44Z Disabled control of system clock
WARNING: no policy specified for kudu/127.3.89.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.3.89.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.3.89.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786289732823074-3431-0/krb5kdc/kudu_127.3.89.254.keytab.
Entry for principal kudu/127.3.89.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786289732823074-3431-0/krb5kdc/kudu_127.3.89.254.keytab.
I20260809 15:39:45.019850 3431 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskm314YR/build/debug/bin/kudu
/tmp/dist-test-taskm314YR/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786289732823074-3431-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786289732823074-3431-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786289732823074-3431-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786289732823074-3431-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.3.89.254:39903
--webserver_interface=127.3.89.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786289732823074-3431-0/krb5kdc/kudu_127.3.89.254.keytab
--principal=kudu/127.3.89.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.3.89.212:37079
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.3.89.254:39903
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786289732823074-3431-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786289732823074-3431-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786289732823074-3431-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 15:39:45.133278 24077 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 15:39:45.133507 24077 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 15:39:45.133544 24077 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 15:39:45.137548 24077 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 15:39:45.137602 24077 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 15:39:45.137621 24077 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 15:39:45.137674 24077 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 15:39:45.137707 24077 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 15:39:45.142777 24077 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.3.89.212:37079
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786289732823074-3431-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786289732823074-3431-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.3.89.254:39903
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786289732823074-3431-0/krb5kdc/kudu_127.3.89.254.keytab
--principal=kudu/127.3.89.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.3.89.254:39903
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786289732823074-3431-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.3.89.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.24077
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786289732823074-3431-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 09 Aug 2026 14:46:55 UTC on ccycloud.quasar-yswkmf.root.comops.site
build id 4576540
I20260809 15:39:45.143762 24077 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 15:39:45.144732 24077 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 15:39:45.152849 24082 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 15:39:45.152904 24077 server_base.cc:1034] running on GCE node
W20260809 15:39:45.152906 24083 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 15:39:45.152853 24085 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 15:39:45.153714 24077 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 15:39:45.154704 24077 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 15:39:45.155903 24077 hybrid_clock.cc:648] HybridClock initialized: now 1786289985155891 us; error 36 us; skew 500 ppm
Aug 09 15:39:45 dist-test-slave-2w3w krb5kdc[24064](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786289985, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.89.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 15:39:45.158448 24077 init.cc:377] Logged in from keytab as kudu/127.3.89.254@KRBTEST.COM (short username kudu)
I20260809 15:39:45.159405 24077 webserver.cc:460] Webserver started at http://127.3.89.254:42791/ using document root <none> and password file <none>
I20260809 15:39:45.159925 24077 fs_manager.cc:359] Metadata directory not provided
I20260809 15:39:45.160013 24077 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 15:39:45.160235 24077 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 15:39:45.162132 24077 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskm314YR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786289732823074-3431-0/minicluster-data/master-0/data/instance:
uuid: "99f9c73b4b6c4c54952df21cc345c7a6"
format_stamp: "Formatted at 2026-08-09 15:39:45 on dist-test-slave-2w3w"
server_key: "886b76f681077338bd66b894ee2de17a"
server_key_iv: "c4ba7479780938364b2ab12ba449be77"
server_key_version: "encryptionkey@0"
I20260809 15:39:45.162681 24077 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskm314YR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786289732823074-3431-0/minicluster-data/master-0/wal/instance:
uuid: "99f9c73b4b6c4c54952df21cc345c7a6"
format_stamp: "Formatted at 2026-08-09 15:39:45 on dist-test-slave-2w3w"
server_key: "886b76f681077338bd66b894ee2de17a"
server_key_iv: "c4ba7479780938364b2ab12ba449be77"
server_key_version: "encryptionkey@0"
I20260809 15:39:45.166198 24077 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.006s sys 0.000s
I20260809 15:39:45.168401 24092 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 15:39:45.169513 24077 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260809 15:39:45.169656 24077 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskm314YR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786289732823074-3431-0/minicluster-data/master-0/data,/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786289732823074-3431-0/minicluster-data/master-0/wal
uuid: "99f9c73b4b6c4c54952df21cc345c7a6"
format_stamp: "Formatted at 2026-08-09 15:39:45 on dist-test-slave-2w3w"
server_key: "886b76f681077338bd66b894ee2de17a"
server_key_iv: "c4ba7479780938364b2ab12ba449be77"
server_key_version: "encryptionkey@0"
I20260809 15:39:45.169761 24077 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskm314YR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786289732823074-3431-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskm314YR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786289732823074-3431-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskm314YR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786289732823074-3431-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 15:39:45.210842 24077 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 15:39:45.213780 24077 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 15:39:45.214007 24077 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 15:39:45.221603 24077 rpc_server.cc:307] RPC server started. Bound to: 127.3.89.254:39903
I20260809 15:39:45.221628 24153 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.3.89.254:39903 every 8 connection(s)
I20260809 15:39:45.222548 24077 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskm314YR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786289732823074-3431-0/minicluster-data/master-0/data/info.pb
I20260809 15:39:45.225247 24154 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 15:39:45.226445 3431 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskm314YR/build/debug/bin/kudu as pid 24077
I20260809 15:39:45.226572 3431 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskm314YR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786289732823074-3431-0/minicluster-data/master-0/wal/instance
I20260809 15:39:45.226859 3431 external_mini_cluster.cc:1428] Setting key a2415cdcab2d5912974c92bec407cb50
I20260809 15:39:45.231905 24154 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 99f9c73b4b6c4c54952df21cc345c7a6: Bootstrap starting.
I20260809 15:39:45.234423 24154 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 99f9c73b4b6c4c54952df21cc345c7a6: Neither blocks nor log segments found. Creating new log.
I20260809 15:39:45.235257 24154 log.cc:824] T 00000000000000000000000000000000 P 99f9c73b4b6c4c54952df21cc345c7a6: Log is configured to *not* fsync() on all Append() calls
Aug 09 15:39:45 dist-test-slave-2w3w krb5kdc[24064](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786289984, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.3.89.254@KRBTEST.COM
I20260809 15:39:45.237363 24154 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 99f9c73b4b6c4c54952df21cc345c7a6: No bootstrap required, opened a new log
I20260809 15:39:45.241817 24157 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:39:45.228267 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52340 (local address 127.3.89.254:39903)
0809 15:39:45.228685 (+ 418us) server_negotiation.cc:206] Beginning negotiation
0809 15:39:45.228704 (+ 19us) server_negotiation.cc:399] Waiting for connection header
0809 15:39:45.228747 (+ 43us) server_negotiation.cc:407] Connection header received
0809 15:39:45.229559 (+ 812us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 15:39:45.229580 (+ 21us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 15:39:45.230002 (+ 422us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 15:39:45.230354 (+ 352us) server_negotiation.cc:226] Negotiated authn=SASL
0809 15:39:45.231045 (+ 691us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:39:45.232467 (+ 1422us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:39:45.233276 (+ 809us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:39:45.233935 (+ 659us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:39:45.234096 (+ 161us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:39:45.236723 (+ 2627us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 15:39:45.236729 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 15:39:45.236743 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 15:39:45.236773 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0809 15:39:45.238698 (+ 1925us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 15:39:45.239184 (+ 486us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 15:39:45.239193 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 15:39:45.239199 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0809 15:39:45.239336 (+ 137us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 15:39:45.239611 (+ 275us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 15:39:45.239615 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 15:39:45.239617 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 15:39:45.240195 (+ 578us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 15:39:45.240389 (+ 194us) server_negotiation.cc:1035] Waiting for connection context
0809 15:39:45.240583 (+ 194us) server_negotiation.cc:299] Negotiation successful
0809 15:39:45.240865 (+ 282us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":275,"thread_start_us":99,"threads_started":1}
I20260809 15:39:45.242017 24154 raft_consensus.cc:348] T 00000000000000000000000000000000 P 99f9c73b4b6c4c54952df21cc345c7a6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "99f9c73b4b6c4c54952df21cc345c7a6" member_type: VOTER last_known_addr { host: "127.3.89.254" port: 39903 } }
I20260809 15:39:45.242254 24154 raft_consensus.cc:374] T 00000000000000000000000000000000 P 99f9c73b4b6c4c54952df21cc345c7a6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 15:39:45.242340 24154 raft_consensus.cc:729] T 00000000000000000000000000000000 P 99f9c73b4b6c4c54952df21cc345c7a6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 99f9c73b4b6c4c54952df21cc345c7a6, State: Initialized, Role: FOLLOWER
I20260809 15:39:45.242755 24154 consensus_queue.cc:260] T 00000000000000000000000000000000 P 99f9c73b4b6c4c54952df21cc345c7a6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "99f9c73b4b6c4c54952df21cc345c7a6" member_type: VOTER last_known_addr { host: "127.3.89.254" port: 39903 } }
I20260809 15:39:45.242874 24154 raft_consensus.cc:388] T 00000000000000000000000000000000 P 99f9c73b4b6c4c54952df21cc345c7a6 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 15:39:45.242987 24154 raft_consensus.cc:482] T 00000000000000000000000000000000 P 99f9c73b4b6c4c54952df21cc345c7a6 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 15:39:45.243091 24154 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 99f9c73b4b6c4c54952df21cc345c7a6 [term 0 FOLLOWER]: Advancing to term 1
I20260809 15:39:45.244151 24154 raft_consensus.cc:504] T 00000000000000000000000000000000 P 99f9c73b4b6c4c54952df21cc345c7a6 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "99f9c73b4b6c4c54952df21cc345c7a6" member_type: VOTER last_known_addr { host: "127.3.89.254" port: 39903 } }
I20260809 15:39:45.244491 24154 leader_election.cc:302] T 00000000000000000000000000000000 P 99f9c73b4b6c4c54952df21cc345c7a6 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 99f9c73b4b6c4c54952df21cc345c7a6; no voters:
I20260809 15:39:45.244743 24154 leader_election.cc:288] T 00000000000000000000000000000000 P 99f9c73b4b6c4c54952df21cc345c7a6 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 15:39:45.244869 24160 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 99f9c73b4b6c4c54952df21cc345c7a6 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 15:39:45.245107 24160 raft_consensus.cc:686] T 00000000000000000000000000000000 P 99f9c73b4b6c4c54952df21cc345c7a6 [term 1 LEADER]: Becoming Leader. State: Replica: 99f9c73b4b6c4c54952df21cc345c7a6, State: Running, Role: LEADER
I20260809 15:39:45.245393 24160 consensus_queue.cc:237] T 00000000000000000000000000000000 P 99f9c73b4b6c4c54952df21cc345c7a6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "99f9c73b4b6c4c54952df21cc345c7a6" member_type: VOTER last_known_addr { host: "127.3.89.254" port: 39903 } }
I20260809 15:39:45.246024 24154 sys_catalog.cc:564] T 00000000000000000000000000000000 P 99f9c73b4b6c4c54952df21cc345c7a6 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 15:39:45.247097 24162 sys_catalog.cc:455] T 00000000000000000000000000000000 P 99f9c73b4b6c4c54952df21cc345c7a6 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 99f9c73b4b6c4c54952df21cc345c7a6. Latest consensus state: current_term: 1 leader_uuid: "99f9c73b4b6c4c54952df21cc345c7a6" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "99f9c73b4b6c4c54952df21cc345c7a6" member_type: VOTER last_known_addr { host: "127.3.89.254" port: 39903 } } }
I20260809 15:39:45.247224 24162 sys_catalog.cc:458] T 00000000000000000000000000000000 P 99f9c73b4b6c4c54952df21cc345c7a6 [sys.catalog]: This master's current role is: LEADER
I20260809 15:39:45.247535 24161 sys_catalog.cc:455] T 00000000000000000000000000000000 P 99f9c73b4b6c4c54952df21cc345c7a6 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "99f9c73b4b6c4c54952df21cc345c7a6" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "99f9c73b4b6c4c54952df21cc345c7a6" member_type: VOTER last_known_addr { host: "127.3.89.254" port: 39903 } } }
I20260809 15:39:45.247638 24161 sys_catalog.cc:458] T 00000000000000000000000000000000 P 99f9c73b4b6c4c54952df21cc345c7a6 [sys.catalog]: This master's current role is: LEADER
I20260809 15:39:45.247732 24169 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 15:39:45.250147 24169 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 15:39:45.255609 24169 catalog_manager.cc:1347] Generated new cluster ID: 04ce3b1337e64caba05d6e10b0d681a3
I20260809 15:39:45.255671 24169 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 15:39:45.265316 24169 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 15:39:45.266458 24169 catalog_manager.cc:1504] Loading token signing keys...
I20260809 15:39:45.277191 24169 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 99f9c73b4b6c4c54952df21cc345c7a6: Generated new TSK 0
I20260809 15:39:45.277765 24169 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.3.89.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.3.89.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.3.89.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786289732823074-3431-0/krb5kdc/kudu_127.3.89.193.keytab.
Entry for principal kudu/127.3.89.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786289732823074-3431-0/krb5kdc/kudu_127.3.89.193.keytab.
I20260809 15:39:45.310282 3431 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskm314YR/build/debug/bin/kudu
/tmp/dist-test-taskm314YR/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786289732823074-3431-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786289732823074-3431-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786289732823074-3431-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786289732823074-3431-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.3.89.193:0
--local_ip_for_outbound_sockets=127.3.89.193
--webserver_interface=127.3.89.193
--webserver_port=0
--tserver_master_addrs=127.3.89.254:39903
--keytab_file=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786289732823074-3431-0/krb5kdc/kudu_127.3.89.193.keytab
--principal=kudu/127.3.89.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.3.89.212:37079
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786289732823074-3431-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786289732823074-3431-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786289732823074-3431-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 15:39:45.421034 24182 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 15:39:45.421274 24182 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 15:39:45.421317 24182 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 15:39:45.425479 24182 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 15:39:45.425540 24182 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 15:39:45.425598 24182 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.3.89.193
I20260809 15:39:45.430692 24182 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.3.89.212:37079
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786289732823074-3431-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786289732823074-3431-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786289732823074-3431-0/krb5kdc/kudu_127.3.89.193.keytab
--principal=kudu/127.3.89.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.3.89.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786289732823074-3431-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.3.89.193
--webserver_port=0
--tserver_master_addrs=127.3.89.254:39903
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.24182
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.3.89.193
--log_dir=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786289732823074-3431-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 09 Aug 2026 14:46:55 UTC on ccycloud.quasar-yswkmf.root.comops.site
build id 4576540
I20260809 15:39:45.431638 24182 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 15:39:45.432540 24182 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260809 15:39:45.441843 24182 server_base.cc:1034] running on GCE node
W20260809 15:39:45.441787 24188 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 15:39:45.441766 24187 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 15:39:45.441797 24192 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 15:39:45.442315 24182 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 15:39:45.442889 24182 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 15:39:45.444047 24182 hybrid_clock.cc:648] HybridClock initialized: now 1786289985444029 us; error 30 us; skew 500 ppm
Aug 09 15:39:45 dist-test-slave-2w3w krb5kdc[24064](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786289985, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.89.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 15:39:45.446866 24182 init.cc:377] Logged in from keytab as kudu/127.3.89.193@KRBTEST.COM (short username kudu)
I20260809 15:39:45.447781 24182 webserver.cc:460] Webserver started at http://127.3.89.193:36611/ using document root <none> and password file <none>
I20260809 15:39:45.448311 24182 fs_manager.cc:359] Metadata directory not provided
I20260809 15:39:45.448371 24182 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 15:39:45.448612 24182 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 15:39:45.450536 24182 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskm314YR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786289732823074-3431-0/minicluster-data/ts-0/data/instance:
uuid: "e8a08c18db504acb978da77ab5769df0"
format_stamp: "Formatted at 2026-08-09 15:39:45 on dist-test-slave-2w3w"
server_key: "89542769a0f656e47fd65ea8800dac4d"
server_key_iv: "32a4a259bd0edd2348ca72d984d0a0af"
server_key_version: "encryptionkey@0"
I20260809 15:39:45.451138 24182 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskm314YR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786289732823074-3431-0/minicluster-data/ts-0/wal/instance:
uuid: "e8a08c18db504acb978da77ab5769df0"
format_stamp: "Formatted at 2026-08-09 15:39:45 on dist-test-slave-2w3w"
server_key: "89542769a0f656e47fd65ea8800dac4d"
server_key_iv: "32a4a259bd0edd2348ca72d984d0a0af"
server_key_version: "encryptionkey@0"
I20260809 15:39:45.454737 24182 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.003s sys 0.000s
I20260809 15:39:45.457067 24199 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 15:39:45.458191 24182 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260809 15:39:45.458314 24182 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskm314YR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786289732823074-3431-0/minicluster-data/ts-0/data,/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786289732823074-3431-0/minicluster-data/ts-0/wal
uuid: "e8a08c18db504acb978da77ab5769df0"
format_stamp: "Formatted at 2026-08-09 15:39:45 on dist-test-slave-2w3w"
server_key: "89542769a0f656e47fd65ea8800dac4d"
server_key_iv: "32a4a259bd0edd2348ca72d984d0a0af"
server_key_version: "encryptionkey@0"
I20260809 15:39:45.458412 24182 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskm314YR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786289732823074-3431-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskm314YR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786289732823074-3431-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskm314YR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786289732823074-3431-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 15:39:45.474315 24182 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 15:39:45.477533 24182 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 15:39:45.477732 24182 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 15:39:45.478310 24182 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 15:39:45.479107 24182 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 15:39:45.479152 24182 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 15:39:45.479219 24182 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 15:39:45.479251 24182 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 15:39:45.487847 24182 rpc_server.cc:307] RPC server started. Bound to: 127.3.89.193:34071
I20260809 15:39:45.487879 24320 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.3.89.193:34071 every 8 connection(s)
I20260809 15:39:45.488679 24182 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskm314YR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786289732823074-3431-0/minicluster-data/ts-0/data/info.pb
I20260809 15:39:45.496634 3431 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskm314YR/build/debug/bin/kudu as pid 24182
I20260809 15:39:45.496800 3431 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskm314YR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786289732823074-3431-0/minicluster-data/ts-0/wal/instance
I20260809 15:39:45.497105 3431 external_mini_cluster.cc:1428] Setting key a37e0d438adc7cce55fc7482aa278667
Aug 09 15:39:45 dist-test-slave-2w3w krb5kdc[24064](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786289985, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.89.193@KRBTEST.COM for kudu/127.3.89.254@KRBTEST.COM
I20260809 15:39:45.501955 24157 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:39:45.490387 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.89.193:60187 (local address 127.3.89.254:39903)
0809 15:39:45.490522 (+ 135us) server_negotiation.cc:206] Beginning negotiation
0809 15:39:45.490526 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 15:39:45.491326 (+ 800us) server_negotiation.cc:407] Connection header received
0809 15:39:45.492590 (+ 1264us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 15:39:45.492593 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 15:39:45.492651 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 15:39:45.492757 (+ 106us) server_negotiation.cc:226] Negotiated authn=SASL
0809 15:39:45.493875 (+ 1118us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:39:45.494417 (+ 542us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:39:45.495593 (+ 1176us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:39:45.495825 (+ 232us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:39:45.495940 (+ 115us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:39:45.498493 (+ 2553us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 15:39:45.498497 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 15:39:45.498500 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 15:39:45.498530 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0809 15:39:45.499919 (+ 1389us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 15:39:45.500447 (+ 528us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 15:39:45.500452 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 15:39:45.500454 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 15:39:45.500535 (+ 81us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 15:39:45.500877 (+ 342us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 15:39:45.500880 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 15:39:45.500882 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 15:39:45.501061 (+ 179us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 15:39:45.501171 (+ 110us) server_negotiation.cc:1035] Waiting for connection context
0809 15:39:45.501657 (+ 486us) server_negotiation.cc:299] Negotiation successful
0809 15:39:45.501786 (+ 129us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":52}
I20260809 15:39:45.502688 24324 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:39:45.490716 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.89.254:39903 (local address 127.3.89.193:60187)
0809 15:39:45.491178 (+ 462us) negotiation.cc:107] Waiting for socket to connect
0809 15:39:45.491220 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0809 15:39:45.492374 (+ 1154us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 15:39:45.492925 (+ 551us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 15:39:45.492931 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 15:39:45.493266 (+ 335us) client_negotiation.cc:189] Negotiated authn=SASL
0809 15:39:45.493720 (+ 454us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:39:45.493733 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:39:45.494554 (+ 821us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:39:45.494559 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:39:45.495452 (+ 893us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:39:45.495470 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:39:45.495981 (+ 511us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:39:45.495985 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:39:45.496164 (+ 179us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:39:45.496810 (+ 646us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 15:39:45.496826 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0809 15:39:45.498328 (+ 1502us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 15:39:45.500040 (+ 1712us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 15:39:45.500046 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 15:39:45.500072 (+ 26us) client_negotiation.cc:760] calling sasl_client_step()
0809 15:39:45.500339 (+ 267us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 15:39:45.500632 (+ 293us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 15:39:45.500635 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 15:39:45.500637 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 15:39:45.500776 (+ 139us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 15:39:45.501187 (+ 411us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 15:39:45.501194 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 15:39:45.501422 (+ 228us) client_negotiation.cc:769] Sending connection context
0809 15:39:45.501615 (+ 193us) client_negotiation.cc:240] Negotiation successful
0809 15:39:45.501914 (+ 299us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":286,"thread_start_us":102,"threads_started":1}
I20260809 15:39:45.503980 24321 heartbeater.cc:347] Connected to a master server at 127.3.89.254:39903
I20260809 15:39:45.504213 24321 heartbeater.cc:464] Registering TS with master...
I20260809 15:39:45.504663 24321 heartbeater.cc:511] Master 127.3.89.254:39903 requested a full tablet report, sending...
I20260809 15:39:45.506075 24111 ts_manager.cc:194] Registered new tserver with Master: e8a08c18db504acb978da77ab5769df0 (127.3.89.193:34071)
I20260809 15:39:45.507251 24111 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.3.89.193@KRBTEST.COM'} at 127.3.89.193:60187
WARNING: no policy specified for kudu/127.3.89.194@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.3.89.194@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.3.89.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786289732823074-3431-0/krb5kdc/kudu_127.3.89.194.keytab.
Entry for principal kudu/127.3.89.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786289732823074-3431-0/krb5kdc/kudu_127.3.89.194.keytab.
I20260809 15:39:45.526008 3431 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskm314YR/build/debug/bin/kudu
/tmp/dist-test-taskm314YR/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786289732823074-3431-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786289732823074-3431-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786289732823074-3431-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786289732823074-3431-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.3.89.194:0
--local_ip_for_outbound_sockets=127.3.89.194
--webserver_interface=127.3.89.194
--webserver_port=0
--tserver_master_addrs=127.3.89.254:39903
--keytab_file=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786289732823074-3431-0/krb5kdc/kudu_127.3.89.194.keytab
--principal=kudu/127.3.89.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.3.89.212:37079
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786289732823074-3431-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786289732823074-3431-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786289732823074-3431-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 15:39:45.635891 24328 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 15:39:45.636178 24328 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 15:39:45.636255 24328 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 15:39:45.640319 24328 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 15:39:45.640414 24328 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 15:39:45.640506 24328 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.3.89.194
I20260809 15:39:45.645985 24328 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.3.89.212:37079
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786289732823074-3431-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786289732823074-3431-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786289732823074-3431-0/krb5kdc/kudu_127.3.89.194.keytab
--principal=kudu/127.3.89.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.3.89.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786289732823074-3431-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.3.89.194
--webserver_port=0
--tserver_master_addrs=127.3.89.254:39903
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.24328
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.3.89.194
--log_dir=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786289732823074-3431-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 09 Aug 2026 14:46:55 UTC on ccycloud.quasar-yswkmf.root.comops.site
build id 4576540
I20260809 15:39:45.647094 24328 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 15:39:45.648195 24328 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260809 15:39:45.657613 24328 server_base.cc:1034] running on GCE node
W20260809 15:39:45.657528 24333 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 15:39:45.657533 24334 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 15:39:45.657527 24337 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 15:39:45.658187 24328 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 15:39:45.658777 24328 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 15:39:45.659956 24328 hybrid_clock.cc:648] HybridClock initialized: now 1786289985659951 us; error 67 us; skew 500 ppm
Aug 09 15:39:45 dist-test-slave-2w3w krb5kdc[24064](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786289985, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.89.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 15:39:45.662724 24328 init.cc:377] Logged in from keytab as kudu/127.3.89.194@KRBTEST.COM (short username kudu)
I20260809 15:39:45.663578 24328 webserver.cc:460] Webserver started at http://127.3.89.194:39121/ using document root <none> and password file <none>
I20260809 15:39:45.664113 24328 fs_manager.cc:359] Metadata directory not provided
I20260809 15:39:45.664206 24328 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 15:39:45.664410 24328 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 15:39:45.666294 24328 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskm314YR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786289732823074-3431-0/minicluster-data/ts-1/data/instance:
uuid: "696cffb8c6ea4ad2aae2be03da3bb9c0"
format_stamp: "Formatted at 2026-08-09 15:39:45 on dist-test-slave-2w3w"
server_key: "be5359d4c9c81ca82fe7512425a3fe0a"
server_key_iv: "32fb634dc246b8fc960693f0551f3d5b"
server_key_version: "encryptionkey@0"
I20260809 15:39:45.666853 24328 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskm314YR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786289732823074-3431-0/minicluster-data/ts-1/wal/instance:
uuid: "696cffb8c6ea4ad2aae2be03da3bb9c0"
format_stamp: "Formatted at 2026-08-09 15:39:45 on dist-test-slave-2w3w"
server_key: "be5359d4c9c81ca82fe7512425a3fe0a"
server_key_iv: "32fb634dc246b8fc960693f0551f3d5b"
server_key_version: "encryptionkey@0"
I20260809 15:39:45.670351 24328 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.002s sys 0.000s
I20260809 15:39:45.672813 24345 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 15:39:45.674019 24328 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260809 15:39:45.674161 24328 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskm314YR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786289732823074-3431-0/minicluster-data/ts-1/data,/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786289732823074-3431-0/minicluster-data/ts-1/wal
uuid: "696cffb8c6ea4ad2aae2be03da3bb9c0"
format_stamp: "Formatted at 2026-08-09 15:39:45 on dist-test-slave-2w3w"
server_key: "be5359d4c9c81ca82fe7512425a3fe0a"
server_key_iv: "32fb634dc246b8fc960693f0551f3d5b"
server_key_version: "encryptionkey@0"
I20260809 15:39:45.674293 24328 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskm314YR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786289732823074-3431-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskm314YR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786289732823074-3431-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskm314YR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786289732823074-3431-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 15:39:45.694487 24328 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 15:39:45.697330 24328 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 15:39:45.697535 24328 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 15:39:45.698135 24328 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 15:39:45.698944 24328 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 15:39:45.699091 24328 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 15:39:45.699159 24328 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 15:39:45.699209 24328 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 15:39:45.708037 24328 rpc_server.cc:307] RPC server started. Bound to: 127.3.89.194:44749
I20260809 15:39:45.708061 24466 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.3.89.194:44749 every 8 connection(s)
I20260809 15:39:45.708928 24328 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskm314YR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786289732823074-3431-0/minicluster-data/ts-1/data/info.pb
I20260809 15:39:45.712476 3431 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskm314YR/build/debug/bin/kudu as pid 24328
I20260809 15:39:45.712607 3431 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskm314YR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786289732823074-3431-0/minicluster-data/ts-1/wal/instance
I20260809 15:39:45.712903 3431 external_mini_cluster.cc:1428] Setting key 947973fee3e2368205cd7b0e0f89d420
Aug 09 15:39:45 dist-test-slave-2w3w krb5kdc[24064](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786289985, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.89.194@KRBTEST.COM for kudu/127.3.89.254@KRBTEST.COM
I20260809 15:39:45.723644 24157 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:39:45.710696 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.89.194:47267 (local address 127.3.89.254:39903)
0809 15:39:45.710845 (+ 149us) server_negotiation.cc:206] Beginning negotiation
0809 15:39:45.710849 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 15:39:45.711743 (+ 894us) server_negotiation.cc:407] Connection header received
0809 15:39:45.713151 (+ 1408us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 15:39:45.713155 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 15:39:45.713213 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 15:39:45.713314 (+ 101us) server_negotiation.cc:226] Negotiated authn=SASL
0809 15:39:45.714652 (+ 1338us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:39:45.715479 (+ 827us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:39:45.716711 (+ 1232us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:39:45.716960 (+ 249us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:39:45.717102 (+ 142us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:39:45.719553 (+ 2451us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 15:39:45.719558 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 15:39:45.719561 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 15:39:45.719593 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0809 15:39:45.721415 (+ 1822us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 15:39:45.721924 (+ 509us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 15:39:45.721927 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 15:39:45.721929 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 15:39:45.722041 (+ 112us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 15:39:45.722397 (+ 356us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 15:39:45.722400 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 15:39:45.722402 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 15:39:45.722643 (+ 241us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 15:39:45.722753 (+ 110us) server_negotiation.cc:1035] Waiting for connection context
0809 15:39:45.723280 (+ 527us) server_negotiation.cc:299] Negotiation successful
0809 15:39:45.723420 (+ 140us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
WARNING: no policy specified for kudu/127.3.89.195@KRBTEST.COM; defaulting to no policy
I20260809 15:39:45.724283 24469 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:39:45.711173 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.89.254:39903 (local address 127.3.89.194:47267)
0809 15:39:45.711587 (+ 414us) negotiation.cc:107] Waiting for socket to connect
0809 15:39:45.711630 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0809 15:39:45.712909 (+ 1279us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 15:39:45.713526 (+ 617us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 15:39:45.713536 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 15:39:45.713934 (+ 398us) client_negotiation.cc:189] Negotiated authn=SASL
0809 15:39:45.714501 (+ 567us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:39:45.714517 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:39:45.715607 (+ 1090us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:39:45.715609 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:39:45.716583 (+ 974us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:39:45.716592 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:39:45.717151 (+ 559us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:39:45.717155 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:39:45.717324 (+ 169us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:39:45.717933 (+ 609us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 15:39:45.717949 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0809 15:39:45.719387 (+ 1438us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 15:39:45.721547 (+ 2160us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 15:39:45.721551 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 15:39:45.721561 (+ 10us) client_negotiation.cc:760] calling sasl_client_step()
0809 15:39:45.721816 (+ 255us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 15:39:45.722143 (+ 327us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 15:39:45.722145 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 15:39:45.722146 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0809 15:39:45.722270 (+ 124us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 15:39:45.722750 (+ 480us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 15:39:45.722754 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 15:39:45.723028 (+ 274us) client_negotiation.cc:769] Sending connection context
0809 15:39:45.723194 (+ 166us) client_negotiation.cc:240] Negotiation successful
0809 15:39:45.723437 (+ 243us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":241,"thread_start_us":102,"threads_started":1}
I20260809 15:39:45.725418 24467 heartbeater.cc:347] Connected to a master server at 127.3.89.254:39903
I20260809 15:39:45.725644 24467 heartbeater.cc:464] Registering TS with master...
I20260809 15:39:45.726230 24467 heartbeater.cc:511] Master 127.3.89.254:39903 requested a full tablet report, sending...
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.3.89.195@KRBTEST.COM" created.
I20260809 15:39:45.727330 24111 ts_manager.cc:194] Registered new tserver with Master: 696cffb8c6ea4ad2aae2be03da3bb9c0 (127.3.89.194:44749)
I20260809 15:39:45.727900 24111 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.3.89.194@KRBTEST.COM'} at 127.3.89.194:47267
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.3.89.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786289732823074-3431-0/krb5kdc/kudu_127.3.89.195.keytab.
Entry for principal kudu/127.3.89.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786289732823074-3431-0/krb5kdc/kudu_127.3.89.195.keytab.
I20260809 15:39:45.742223 3431 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskm314YR/build/debug/bin/kudu
/tmp/dist-test-taskm314YR/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786289732823074-3431-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786289732823074-3431-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786289732823074-3431-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786289732823074-3431-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.3.89.195:0
--local_ip_for_outbound_sockets=127.3.89.195
--webserver_interface=127.3.89.195
--webserver_port=0
--tserver_master_addrs=127.3.89.254:39903
--keytab_file=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786289732823074-3431-0/krb5kdc/kudu_127.3.89.195.keytab
--principal=kudu/127.3.89.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.3.89.212:37079
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786289732823074-3431-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786289732823074-3431-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786289732823074-3431-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 15:39:45.851980 24473 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 15:39:45.852268 24473 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 15:39:45.852349 24473 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 15:39:45.856420 24473 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 15:39:45.856503 24473 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 15:39:45.856590 24473 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.3.89.195
I20260809 15:39:45.861768 24473 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.3.89.212:37079
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786289732823074-3431-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786289732823074-3431-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786289732823074-3431-0/krb5kdc/kudu_127.3.89.195.keytab
--principal=kudu/127.3.89.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.3.89.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786289732823074-3431-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.3.89.195
--webserver_port=0
--tserver_master_addrs=127.3.89.254:39903
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.24473
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.3.89.195
--log_dir=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786289732823074-3431-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 09 Aug 2026 14:46:55 UTC on ccycloud.quasar-yswkmf.root.comops.site
build id 4576540
I20260809 15:39:45.862850 24473 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 15:39:45.863862 24473 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 15:39:45.873513 24478 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 15:39:45.873481 24479 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 15:39:45.873515 24481 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 15:39:45.873903 24473 server_base.cc:1034] running on GCE node
I20260809 15:39:45.874280 24473 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 15:39:45.874908 24473 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 15:39:45.876134 24473 hybrid_clock.cc:648] HybridClock initialized: now 1786289985876105 us; error 41 us; skew 500 ppm
Aug 09 15:39:45 dist-test-slave-2w3w krb5kdc[24064](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786289985, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.89.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 15:39:45.879168 24473 init.cc:377] Logged in from keytab as kudu/127.3.89.195@KRBTEST.COM (short username kudu)
I20260809 15:39:45.879987 24473 webserver.cc:460] Webserver started at http://127.3.89.195:43159/ using document root <none> and password file <none>
I20260809 15:39:45.880533 24473 fs_manager.cc:359] Metadata directory not provided
I20260809 15:39:45.880628 24473 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 15:39:45.880860 24473 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 15:39:45.882824 24473 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskm314YR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786289732823074-3431-0/minicluster-data/ts-2/data/instance:
uuid: "3297ca45214544db92252a24690e352e"
format_stamp: "Formatted at 2026-08-09 15:39:45 on dist-test-slave-2w3w"
server_key: "ba942b93c27bd1fd7f17148b682db2ef"
server_key_iv: "6ba9a796a770116fcd5b4a4c35a47b4c"
server_key_version: "encryptionkey@0"
I20260809 15:39:45.883461 24473 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskm314YR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786289732823074-3431-0/minicluster-data/ts-2/wal/instance:
uuid: "3297ca45214544db92252a24690e352e"
format_stamp: "Formatted at 2026-08-09 15:39:45 on dist-test-slave-2w3w"
server_key: "ba942b93c27bd1fd7f17148b682db2ef"
server_key_iv: "6ba9a796a770116fcd5b4a4c35a47b4c"
server_key_version: "encryptionkey@0"
I20260809 15:39:45.886948 24473 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.003s sys 0.000s
I20260809 15:39:45.889395 24488 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 15:39:45.890513 24473 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.000s sys 0.002s
I20260809 15:39:45.890650 24473 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskm314YR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786289732823074-3431-0/minicluster-data/ts-2/data,/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786289732823074-3431-0/minicluster-data/ts-2/wal
uuid: "3297ca45214544db92252a24690e352e"
format_stamp: "Formatted at 2026-08-09 15:39:45 on dist-test-slave-2w3w"
server_key: "ba942b93c27bd1fd7f17148b682db2ef"
server_key_iv: "6ba9a796a770116fcd5b4a4c35a47b4c"
server_key_version: "encryptionkey@0"
I20260809 15:39:45.890755 24473 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskm314YR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786289732823074-3431-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskm314YR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786289732823074-3431-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskm314YR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786289732823074-3431-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 15:39:45.907541 24473 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 15:39:45.910229 24473 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 15:39:45.910434 24473 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 15:39:45.911108 24473 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 15:39:45.911921 24473 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 15:39:45.911990 24473 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 15:39:45.912060 24473 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 15:39:45.912096 24473 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 15:39:45.921205 24473 rpc_server.cc:307] RPC server started. Bound to: 127.3.89.195:44119
I20260809 15:39:45.921233 24612 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.3.89.195:44119 every 8 connection(s)
I20260809 15:39:45.922040 24473 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskm314YR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786289732823074-3431-0/minicluster-data/ts-2/data/info.pb
I20260809 15:39:45.928614 3431 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskm314YR/build/debug/bin/kudu as pid 24473
I20260809 15:39:45.928761 3431 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskm314YR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786289732823074-3431-0/minicluster-data/ts-2/wal/instance
I20260809 15:39:45.929102 3431 external_mini_cluster.cc:1428] Setting key 90be01b9e851fbd7553d3ea1420798c5
Aug 09 15:39:45 dist-test-slave-2w3w krb5kdc[24064](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786289985, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.89.195@KRBTEST.COM for kudu/127.3.89.254@KRBTEST.COM
I20260809 15:39:45.935740 24157 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:39:45.923748 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.89.195:57409 (local address 127.3.89.254:39903)
0809 15:39:45.923877 (+ 129us) server_negotiation.cc:206] Beginning negotiation
0809 15:39:45.923881 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 15:39:45.924669 (+ 788us) server_negotiation.cc:407] Connection header received
0809 15:39:45.925886 (+ 1217us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 15:39:45.925889 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 15:39:45.925935 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 15:39:45.926012 (+ 77us) server_negotiation.cc:226] Negotiated authn=SASL
0809 15:39:45.927290 (+ 1278us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:39:45.927823 (+ 533us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:39:45.929034 (+ 1211us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:39:45.929316 (+ 282us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:39:45.929434 (+ 118us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:39:45.932236 (+ 2802us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 15:39:45.932239 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 15:39:45.932241 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 15:39:45.932265 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0809 15:39:45.933838 (+ 1573us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 15:39:45.934374 (+ 536us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 15:39:45.934377 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 15:39:45.934378 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 15:39:45.934465 (+ 87us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 15:39:45.934773 (+ 308us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 15:39:45.934775 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 15:39:45.934776 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 15:39:45.934950 (+ 174us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 15:39:45.935105 (+ 155us) server_negotiation.cc:1035] Waiting for connection context
0809 15:39:45.935525 (+ 420us) server_negotiation.cc:299] Negotiation successful
0809 15:39:45.935625 (+ 100us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":48}
I20260809 15:39:45.936551 24617 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:39:45.924088 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.89.254:39903 (local address 127.3.89.195:57409)
0809 15:39:45.924520 (+ 432us) negotiation.cc:107] Waiting for socket to connect
0809 15:39:45.924561 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0809 15:39:45.925687 (+ 1126us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 15:39:45.926195 (+ 508us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 15:39:45.926203 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 15:39:45.926641 (+ 438us) client_negotiation.cc:189] Negotiated authn=SASL
0809 15:39:45.927153 (+ 512us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:39:45.927169 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:39:45.927933 (+ 764us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:39:45.927935 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:39:45.928892 (+ 957us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:39:45.928902 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:39:45.929484 (+ 582us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:39:45.929488 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:39:45.929687 (+ 199us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:39:45.930323 (+ 636us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 15:39:45.930342 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0809 15:39:45.932058 (+ 1716us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 15:39:45.933946 (+ 1888us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 15:39:45.933951 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 15:39:45.933966 (+ 15us) client_negotiation.cc:760] calling sasl_client_step()
0809 15:39:45.934239 (+ 273us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 15:39:45.934555 (+ 316us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 15:39:45.934557 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 15:39:45.934558 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0809 15:39:45.934683 (+ 125us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 15:39:45.935102 (+ 419us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 15:39:45.935106 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 15:39:45.935315 (+ 209us) client_negotiation.cc:769] Sending connection context
0809 15:39:45.935471 (+ 156us) client_negotiation.cc:240] Negotiation successful
0809 15:39:45.935703 (+ 232us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":249,"thread_start_us":111,"threads_started":1}
I20260809 15:39:45.937660 24613 heartbeater.cc:347] Connected to a master server at 127.3.89.254:39903
I20260809 15:39:45.937887 24613 heartbeater.cc:464] Registering TS with master...
I20260809 15:39:45.938345 24613 heartbeater.cc:511] Master 127.3.89.254:39903 requested a full tablet report, sending...
I20260809 15:39:45.939391 24111 ts_manager.cc:194] Registered new tserver with Master: 3297ca45214544db92252a24690e352e (127.3.89.195:44119)
I20260809 15:39:45.939981 24111 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.3.89.195@KRBTEST.COM'} at 127.3.89.195:57409
I20260809 15:39:45.943066 3431 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260809 15:39:45.952557 24157 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:39:45.945010 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52344 (local address 127.3.89.254:39903)
0809 15:39:45.945157 (+ 147us) server_negotiation.cc:206] Beginning negotiation
0809 15:39:45.945161 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 15:39:45.945347 (+ 186us) server_negotiation.cc:407] Connection header received
0809 15:39:45.945545 (+ 198us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 15:39:45.945548 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 15:39:45.945594 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 15:39:45.945680 (+ 86us) server_negotiation.cc:226] Negotiated authn=SASL
0809 15:39:45.946391 (+ 711us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:39:45.947030 (+ 639us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:39:45.947959 (+ 929us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:39:45.948192 (+ 233us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:39:45.948302 (+ 110us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:39:45.949575 (+ 1273us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 15:39:45.949579 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 15:39:45.949581 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 15:39:45.949623 (+ 42us) server_negotiation.cc:922] calling sasl_server_start()
0809 15:39:45.950870 (+ 1247us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 15:39:45.951321 (+ 451us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 15:39:45.951325 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 15:39:45.951327 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 15:39:45.951409 (+ 82us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 15:39:45.951680 (+ 271us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 15:39:45.951684 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 15:39:45.951686 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 15:39:45.951902 (+ 216us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 15:39:45.952038 (+ 136us) server_negotiation.cc:1035] Waiting for connection context
0809 15:39:45.952278 (+ 240us) server_negotiation.cc:299] Negotiation successful
0809 15:39:45.952400 (+ 122us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":51}
I20260809 15:39:45.955174 24111 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:52344:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260809 15:39:45.958051 24111 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260809 15:39:45.972786 24632 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:39:45.967351 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.89.193:34071 (local address 127.0.0.1:47910)
0809 15:39:45.967823 (+ 472us) negotiation.cc:107] Waiting for socket to connect
0809 15:39:45.967840 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0809 15:39:45.967933 (+ 93us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 15:39:45.968530 (+ 597us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 15:39:45.968536 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 15:39:45.968557 (+ 21us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 15:39:45.968717 (+ 160us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:39:45.968725 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:39:45.970249 (+ 1524us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:39:45.970252 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:39:45.971269 (+ 1017us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:39:45.971279 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:39:45.972391 (+ 1112us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:39:45.972397 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:39:45.972480 (+ 83us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:39:45.972500 (+ 20us) client_negotiation.cc:769] Sending connection context
0809 15:39:45.972570 (+ 70us) client_negotiation.cc:240] Negotiation successful
0809 15:39:45.972622 (+ 52us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":406,"mutex_wait_us":150,"spinlock_wait_cycles":1792,"thread_start_us":83,"threads_started":1}
I20260809 15:39:45.972978 24630 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:39:45.967020 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.89.195:44119 (local address 127.0.0.1:47256)
0809 15:39:45.967527 (+ 507us) negotiation.cc:107] Waiting for socket to connect
0809 15:39:45.967541 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0809 15:39:45.967619 (+ 78us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 15:39:45.968555 (+ 936us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 15:39:45.968558 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 15:39:45.968570 (+ 12us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 15:39:45.968730 (+ 160us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:39:45.968740 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:39:45.970232 (+ 1492us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:39:45.970243 (+ 11us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:39:45.971451 (+ 1208us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:39:45.971461 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:39:45.972490 (+ 1029us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:39:45.972494 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:39:45.972624 (+ 130us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:39:45.972648 (+ 24us) client_negotiation.cc:769] Sending connection context
0809 15:39:45.972700 (+ 52us) client_negotiation.cc:240] Negotiation successful
0809 15:39:45.972764 (+ 64us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":432,"mutex_wait_us":45,"thread_start_us":106,"threads_started":1}
I20260809 15:39:45.973237 24633 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:39:45.967751 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47910 (local address 127.3.89.193:34071)
0809 15:39:45.968055 (+ 304us) server_negotiation.cc:206] Beginning negotiation
0809 15:39:45.968072 (+ 17us) server_negotiation.cc:399] Waiting for connection header
0809 15:39:45.968094 (+ 22us) server_negotiation.cc:407] Connection header received
0809 15:39:45.968165 (+ 71us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 15:39:45.968170 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 15:39:45.968356 (+ 186us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 15:39:45.968517 (+ 161us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 15:39:45.968862 (+ 345us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:39:45.970031 (+ 1169us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:39:45.971400 (+ 1369us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:39:45.972230 (+ 830us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:39:45.972343 (+ 113us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:39:45.972511 (+ 168us) server_negotiation.cc:1035] Waiting for connection context
0809 15:39:45.972600 (+ 89us) server_negotiation.cc:299] Negotiation successful
0809 15:39:45.972708 (+ 108us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":199,"thread_start_us":94,"threads_started":1}
I20260809 15:39:45.973486 24629 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:39:45.966398 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39522 (local address 127.3.89.194:44749)
0809 15:39:45.966707 (+ 309us) server_negotiation.cc:206] Beginning negotiation
0809 15:39:45.966722 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0809 15:39:45.967596 (+ 874us) server_negotiation.cc:407] Connection header received
0809 15:39:45.967684 (+ 88us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 15:39:45.967690 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 15:39:45.967891 (+ 201us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 15:39:45.968047 (+ 156us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 15:39:45.968570 (+ 523us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:39:45.969492 (+ 922us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:39:45.971412 (+ 1920us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:39:45.972026 (+ 614us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:39:45.972155 (+ 129us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:39:45.972310 (+ 155us) server_negotiation.cc:1035] Waiting for connection context
0809 15:39:45.973250 (+ 940us) server_negotiation.cc:299] Negotiation successful
0809 15:39:45.973345 (+ 95us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":212,"thread_start_us":134,"threads_started":1}
I20260809 15:39:45.973565 24631 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:39:45.967187 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47256 (local address 127.3.89.195:44119)
0809 15:39:45.968169 (+ 982us) server_negotiation.cc:206] Beginning negotiation
0809 15:39:45.968178 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0809 15:39:45.968195 (+ 17us) server_negotiation.cc:407] Connection header received
0809 15:39:45.968253 (+ 58us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 15:39:45.968257 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 15:39:45.968390 (+ 133us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 15:39:45.968714 (+ 324us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 15:39:45.968876 (+ 162us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:39:45.970031 (+ 1155us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:39:45.971567 (+ 1536us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:39:45.972352 (+ 785us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:39:45.973133 (+ 781us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:39:45.973281 (+ 148us) server_negotiation.cc:1035] Waiting for connection context
0809 15:39:45.973340 (+ 59us) server_negotiation.cc:299] Negotiation successful
0809 15:39:45.973429 (+ 89us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":901,"thread_start_us":140,"threads_started":1}
I20260809 15:39:45.973759 24628 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:39:45.966214 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.89.194:44749 (local address 127.0.0.1:39522)
0809 15:39:45.967228 (+ 1014us) negotiation.cc:107] Waiting for socket to connect
0809 15:39:45.967279 (+ 51us) client_negotiation.cc:174] Beginning negotiation
0809 15:39:45.967431 (+ 152us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 15:39:45.968249 (+ 818us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 15:39:45.968255 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 15:39:45.968277 (+ 22us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 15:39:45.968422 (+ 145us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:39:45.968433 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:39:45.969645 (+ 1212us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:39:45.969650 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:39:45.971268 (+ 1618us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:39:45.971285 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:39:45.972155 (+ 870us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:39:45.972158 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:39:45.972282 (+ 124us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:39:45.972323 (+ 41us) client_negotiation.cc:769] Sending connection context
0809 15:39:45.972430 (+ 107us) client_negotiation.cc:240] Negotiation successful
0809 15:39:45.972554 (+ 124us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":847,"mutex_wait_us":41,"thread_start_us":105,"threads_started":1}
I20260809 15:39:45.976122 24538 tablet_service.cc:1467] Processing CreateTablet for tablet 4369c8d6ef1b413bb904bb0481ab3f77 (DEFAULT_TABLE table=test-table [id=bab709d3838b4542bbb6d19064adb2c0]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 15:39:45.976125 24394 tablet_service.cc:1467] Processing CreateTablet for tablet 4369c8d6ef1b413bb904bb0481ab3f77 (DEFAULT_TABLE table=test-table [id=bab709d3838b4542bbb6d19064adb2c0]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 15:39:45.977167 24394 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 4369c8d6ef1b413bb904bb0481ab3f77. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 15:39:45.977176 24538 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 4369c8d6ef1b413bb904bb0481ab3f77. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 15:39:45.978427 24251 tablet_service.cc:1467] Processing CreateTablet for tablet 4369c8d6ef1b413bb904bb0481ab3f77 (DEFAULT_TABLE table=test-table [id=bab709d3838b4542bbb6d19064adb2c0]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 15:39:45.979368 24251 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 4369c8d6ef1b413bb904bb0481ab3f77. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 15:39:45.983356 24634 tablet_bootstrap.cc:492] T 4369c8d6ef1b413bb904bb0481ab3f77 P 696cffb8c6ea4ad2aae2be03da3bb9c0: Bootstrap starting.
I20260809 15:39:45.985383 24634 tablet_bootstrap.cc:654] T 4369c8d6ef1b413bb904bb0481ab3f77 P 696cffb8c6ea4ad2aae2be03da3bb9c0: Neither blocks nor log segments found. Creating new log.
I20260809 15:39:45.985929 24637 tablet_bootstrap.cc:492] T 4369c8d6ef1b413bb904bb0481ab3f77 P e8a08c18db504acb978da77ab5769df0: Bootstrap starting.
I20260809 15:39:45.986476 24636 tablet_bootstrap.cc:492] T 4369c8d6ef1b413bb904bb0481ab3f77 P 3297ca45214544db92252a24690e352e: Bootstrap starting.
I20260809 15:39:45.986552 24634 log.cc:824] T 4369c8d6ef1b413bb904bb0481ab3f77 P 696cffb8c6ea4ad2aae2be03da3bb9c0: Log is configured to *not* fsync() on all Append() calls
I20260809 15:39:45.987996 24637 tablet_bootstrap.cc:654] T 4369c8d6ef1b413bb904bb0481ab3f77 P e8a08c18db504acb978da77ab5769df0: Neither blocks nor log segments found. Creating new log.
I20260809 15:39:45.988474 24634 tablet_bootstrap.cc:492] T 4369c8d6ef1b413bb904bb0481ab3f77 P 696cffb8c6ea4ad2aae2be03da3bb9c0: No bootstrap required, opened a new log
I20260809 15:39:45.988483 24636 tablet_bootstrap.cc:654] T 4369c8d6ef1b413bb904bb0481ab3f77 P 3297ca45214544db92252a24690e352e: Neither blocks nor log segments found. Creating new log.
I20260809 15:39:45.988674 24634 ts_tablet_manager.cc:1397] T 4369c8d6ef1b413bb904bb0481ab3f77 P 696cffb8c6ea4ad2aae2be03da3bb9c0: Time spent bootstrapping tablet: real 0.005s user 0.000s sys 0.004s
I20260809 15:39:45.989257 24637 log.cc:824] T 4369c8d6ef1b413bb904bb0481ab3f77 P e8a08c18db504acb978da77ab5769df0: Log is configured to *not* fsync() on all Append() calls
I20260809 15:39:45.989612 24636 log.cc:824] T 4369c8d6ef1b413bb904bb0481ab3f77 P 3297ca45214544db92252a24690e352e: Log is configured to *not* fsync() on all Append() calls
I20260809 15:39:45.991073 24637 tablet_bootstrap.cc:492] T 4369c8d6ef1b413bb904bb0481ab3f77 P e8a08c18db504acb978da77ab5769df0: No bootstrap required, opened a new log
I20260809 15:39:45.991261 24637 ts_tablet_manager.cc:1397] T 4369c8d6ef1b413bb904bb0481ab3f77 P e8a08c18db504acb978da77ab5769df0: Time spent bootstrapping tablet: real 0.005s user 0.005s sys 0.000s
I20260809 15:39:45.991474 24636 tablet_bootstrap.cc:492] T 4369c8d6ef1b413bb904bb0481ab3f77 P 3297ca45214544db92252a24690e352e: No bootstrap required, opened a new log
I20260809 15:39:45.991664 24636 ts_tablet_manager.cc:1397] T 4369c8d6ef1b413bb904bb0481ab3f77 P 3297ca45214544db92252a24690e352e: Time spent bootstrapping tablet: real 0.005s user 0.004s sys 0.000s
I20260809 15:39:45.992280 24634 raft_consensus.cc:348] T 4369c8d6ef1b413bb904bb0481ab3f77 P 696cffb8c6ea4ad2aae2be03da3bb9c0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "696cffb8c6ea4ad2aae2be03da3bb9c0" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 44749 } } peers { permanent_uuid: "3297ca45214544db92252a24690e352e" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 44119 } } peers { permanent_uuid: "e8a08c18db504acb978da77ab5769df0" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 34071 } }
I20260809 15:39:45.992554 24634 raft_consensus.cc:374] T 4369c8d6ef1b413bb904bb0481ab3f77 P 696cffb8c6ea4ad2aae2be03da3bb9c0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 15:39:45.992619 24634 raft_consensus.cc:729] T 4369c8d6ef1b413bb904bb0481ab3f77 P 696cffb8c6ea4ad2aae2be03da3bb9c0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 696cffb8c6ea4ad2aae2be03da3bb9c0, State: Initialized, Role: FOLLOWER
I20260809 15:39:45.993038 24634 consensus_queue.cc:260] T 4369c8d6ef1b413bb904bb0481ab3f77 P 696cffb8c6ea4ad2aae2be03da3bb9c0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "696cffb8c6ea4ad2aae2be03da3bb9c0" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 44749 } } peers { permanent_uuid: "3297ca45214544db92252a24690e352e" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 44119 } } peers { permanent_uuid: "e8a08c18db504acb978da77ab5769df0" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 34071 } }
I20260809 15:39:45.993672 24467 heartbeater.cc:503] Master 127.3.89.254:39903 was elected leader, sending a full tablet report...
I20260809 15:39:45.994148 24634 ts_tablet_manager.cc:1428] T 4369c8d6ef1b413bb904bb0481ab3f77 P 696cffb8c6ea4ad2aae2be03da3bb9c0: Time spent starting tablet: real 0.005s user 0.005s sys 0.002s
I20260809 15:39:45.994697 24637 raft_consensus.cc:348] T 4369c8d6ef1b413bb904bb0481ab3f77 P e8a08c18db504acb978da77ab5769df0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "696cffb8c6ea4ad2aae2be03da3bb9c0" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 44749 } } peers { permanent_uuid: "3297ca45214544db92252a24690e352e" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 44119 } } peers { permanent_uuid: "e8a08c18db504acb978da77ab5769df0" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 34071 } }
I20260809 15:39:45.994997 24637 raft_consensus.cc:374] T 4369c8d6ef1b413bb904bb0481ab3f77 P e8a08c18db504acb978da77ab5769df0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 15:39:45.995067 24637 raft_consensus.cc:729] T 4369c8d6ef1b413bb904bb0481ab3f77 P e8a08c18db504acb978da77ab5769df0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e8a08c18db504acb978da77ab5769df0, State: Initialized, Role: FOLLOWER
I20260809 15:39:45.995515 24637 consensus_queue.cc:260] T 4369c8d6ef1b413bb904bb0481ab3f77 P e8a08c18db504acb978da77ab5769df0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "696cffb8c6ea4ad2aae2be03da3bb9c0" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 44749 } } peers { permanent_uuid: "3297ca45214544db92252a24690e352e" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 44119 } } peers { permanent_uuid: "e8a08c18db504acb978da77ab5769df0" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 34071 } }
I20260809 15:39:45.995718 24642 raft_consensus.cc:482] T 4369c8d6ef1b413bb904bb0481ab3f77 P e8a08c18db504acb978da77ab5769df0 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 15:39:45.995913 24642 raft_consensus.cc:504] T 4369c8d6ef1b413bb904bb0481ab3f77 P e8a08c18db504acb978da77ab5769df0 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "696cffb8c6ea4ad2aae2be03da3bb9c0" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 44749 } } peers { permanent_uuid: "3297ca45214544db92252a24690e352e" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 44119 } } peers { permanent_uuid: "e8a08c18db504acb978da77ab5769df0" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 34071 } }
I20260809 15:39:45.996169 24636 raft_consensus.cc:348] T 4369c8d6ef1b413bb904bb0481ab3f77 P 3297ca45214544db92252a24690e352e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "696cffb8c6ea4ad2aae2be03da3bb9c0" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 44749 } } peers { permanent_uuid: "3297ca45214544db92252a24690e352e" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 44119 } } peers { permanent_uuid: "e8a08c18db504acb978da77ab5769df0" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 34071 } }
I20260809 15:39:45.996426 24636 raft_consensus.cc:374] T 4369c8d6ef1b413bb904bb0481ab3f77 P 3297ca45214544db92252a24690e352e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 15:39:45.996456 24637 ts_tablet_manager.cc:1428] T 4369c8d6ef1b413bb904bb0481ab3f77 P e8a08c18db504acb978da77ab5769df0: Time spent starting tablet: real 0.005s user 0.005s sys 0.000s
I20260809 15:39:45.996487 24636 raft_consensus.cc:729] T 4369c8d6ef1b413bb904bb0481ab3f77 P 3297ca45214544db92252a24690e352e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3297ca45214544db92252a24690e352e, State: Initialized, Role: FOLLOWER
I20260809 15:39:45.996843 24321 heartbeater.cc:503] Master 127.3.89.254:39903 was elected leader, sending a full tablet report...
I20260809 15:39:45.996879 24636 consensus_queue.cc:260] T 4369c8d6ef1b413bb904bb0481ab3f77 P 3297ca45214544db92252a24690e352e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "696cffb8c6ea4ad2aae2be03da3bb9c0" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 44749 } } peers { permanent_uuid: "3297ca45214544db92252a24690e352e" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 44119 } } peers { permanent_uuid: "e8a08c18db504acb978da77ab5769df0" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 34071 } }
I20260809 15:39:45.997519 24613 heartbeater.cc:503] Master 127.3.89.254:39903 was elected leader, sending a full tablet report...
I20260809 15:39:45.997778 24636 ts_tablet_manager.cc:1428] T 4369c8d6ef1b413bb904bb0481ab3f77 P 3297ca45214544db92252a24690e352e: Time spent starting tablet: real 0.006s user 0.005s sys 0.000s
I20260809 15:39:45.999562 24642 leader_election.cc:288] T 4369c8d6ef1b413bb904bb0481ab3f77 P e8a08c18db504acb978da77ab5769df0 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 696cffb8c6ea4ad2aae2be03da3bb9c0 (127.3.89.194:44749), 3297ca45214544db92252a24690e352e (127.3.89.195:44119)
I20260809 15:39:46.002310 24629 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:39:45.998108 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.89.193:39539 (local address 127.3.89.194:44749)
0809 15:39:45.998212 (+ 104us) server_negotiation.cc:206] Beginning negotiation
0809 15:39:45.998216 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 15:39:45.998229 (+ 13us) server_negotiation.cc:407] Connection header received
0809 15:39:45.998270 (+ 41us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 15:39:45.998273 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 15:39:45.998321 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 15:39:45.998412 (+ 91us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 15:39:45.998805 (+ 393us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:39:45.999575 (+ 770us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:39:46.001104 (+ 1529us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:39:46.001731 (+ 627us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:39:46.001864 (+ 133us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:39:46.001913 (+ 49us) server_negotiation.cc:1035] Waiting for connection context
0809 15:39:46.002132 (+ 219us) server_negotiation.cc:299] Negotiation successful
0809 15:39:46.002200 (+ 68us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":41}
I20260809 15:39:46.002310 24324 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:39:45.997537 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.89.194:44749 (local address 127.3.89.193:39539)
0809 15:39:45.997814 (+ 277us) negotiation.cc:107] Waiting for socket to connect
0809 15:39:45.997831 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0809 15:39:45.997921 (+ 90us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 15:39:45.998541 (+ 620us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 15:39:45.998544 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 15:39:45.998561 (+ 17us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 15:39:45.998687 (+ 126us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:39:45.998694 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:39:45.999753 (+ 1059us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:39:45.999756 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:39:46.000940 (+ 1184us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:39:46.000952 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:39:46.001868 (+ 916us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:39:46.001872 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:39:46.002007 (+ 135us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:39:46.002041 (+ 34us) client_negotiation.cc:769] Sending connection context
0809 15:39:46.002116 (+ 75us) client_negotiation.cc:240] Negotiation successful
0809 15:39:46.002188 (+ 72us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":182}
I20260809 15:39:46.002907 24416 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4369c8d6ef1b413bb904bb0481ab3f77" candidate_uuid: "e8a08c18db504acb978da77ab5769df0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "696cffb8c6ea4ad2aae2be03da3bb9c0" is_pre_election: true
I20260809 15:39:46.003183 24416 raft_consensus.cc:2454] T 4369c8d6ef1b413bb904bb0481ab3f77 P 696cffb8c6ea4ad2aae2be03da3bb9c0 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e8a08c18db504acb978da77ab5769df0 in term 0.
I20260809 15:39:46.003645 24201 leader_election.cc:302] T 4369c8d6ef1b413bb904bb0481ab3f77 P e8a08c18db504acb978da77ab5769df0 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 696cffb8c6ea4ad2aae2be03da3bb9c0, e8a08c18db504acb978da77ab5769df0; no voters:
I20260809 15:39:46.003873 24642 raft_consensus.cc:2781] T 4369c8d6ef1b413bb904bb0481ab3f77 P e8a08c18db504acb978da77ab5769df0 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260809 15:39:46.003961 24642 raft_consensus.cc:482] T 4369c8d6ef1b413bb904bb0481ab3f77 P e8a08c18db504acb978da77ab5769df0 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 15:39:46.004029 24642 raft_consensus.cc:3037] T 4369c8d6ef1b413bb904bb0481ab3f77 P e8a08c18db504acb978da77ab5769df0 [term 0 FOLLOWER]: Advancing to term 1
I20260809 15:39:46.004415 24631 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:39:45.999423 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.89.193:36343 (local address 127.3.89.195:44119)
0809 15:39:45.999521 (+ 98us) server_negotiation.cc:206] Beginning negotiation
0809 15:39:45.999526 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 15:39:46.000214 (+ 688us) server_negotiation.cc:407] Connection header received
0809 15:39:46.000417 (+ 203us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 15:39:46.000420 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 15:39:46.000463 (+ 43us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 15:39:46.000511 (+ 48us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 15:39:46.000825 (+ 314us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:39:46.001510 (+ 685us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:39:46.003335 (+ 1825us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:39:46.003787 (+ 452us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:39:46.003924 (+ 137us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:39:46.003988 (+ 64us) server_negotiation.cc:1035] Waiting for connection context
0809 15:39:46.004245 (+ 257us) server_negotiation.cc:299] Negotiation successful
0809 15:39:46.004314 (+ 69us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":39}
I20260809 15:39:46.004438 24645 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:39:45.999117 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.89.195:44119 (local address 127.3.89.193:36343)
0809 15:39:46.000130 (+ 1013us) negotiation.cc:107] Waiting for socket to connect
0809 15:39:46.000145 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0809 15:39:46.000295 (+ 150us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 15:39:46.000613 (+ 318us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 15:39:46.000615 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 15:39:46.000628 (+ 13us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 15:39:46.000714 (+ 86us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:39:46.000718 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:39:46.001643 (+ 925us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:39:46.001647 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:39:46.003185 (+ 1538us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:39:46.003198 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:39:46.003940 (+ 742us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:39:46.003944 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:39:46.004105 (+ 161us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:39:46.004128 (+ 23us) client_negotiation.cc:769] Sending connection context
0809 15:39:46.004228 (+ 100us) client_negotiation.cc:240] Negotiation successful
0809 15:39:46.004296 (+ 68us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":942,"thread_start_us":84,"threads_started":1}
I20260809 15:39:46.004913 24561 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4369c8d6ef1b413bb904bb0481ab3f77" candidate_uuid: "e8a08c18db504acb978da77ab5769df0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3297ca45214544db92252a24690e352e" is_pre_election: true
I20260809 15:39:46.005144 24561 raft_consensus.cc:2454] T 4369c8d6ef1b413bb904bb0481ab3f77 P 3297ca45214544db92252a24690e352e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e8a08c18db504acb978da77ab5769df0 in term 0.
I20260809 15:39:46.005894 24642 raft_consensus.cc:504] T 4369c8d6ef1b413bb904bb0481ab3f77 P e8a08c18db504acb978da77ab5769df0 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "696cffb8c6ea4ad2aae2be03da3bb9c0" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 44749 } } peers { permanent_uuid: "3297ca45214544db92252a24690e352e" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 44119 } } peers { permanent_uuid: "e8a08c18db504acb978da77ab5769df0" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 34071 } }
I20260809 15:39:46.006203 24642 leader_election.cc:288] T 4369c8d6ef1b413bb904bb0481ab3f77 P e8a08c18db504acb978da77ab5769df0 [CANDIDATE]: Term 1 election: Requested vote from peers 696cffb8c6ea4ad2aae2be03da3bb9c0 (127.3.89.194:44749), 3297ca45214544db92252a24690e352e (127.3.89.195:44119)
I20260809 15:39:46.006568 24416 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4369c8d6ef1b413bb904bb0481ab3f77" candidate_uuid: "e8a08c18db504acb978da77ab5769df0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "696cffb8c6ea4ad2aae2be03da3bb9c0"
I20260809 15:39:46.006707 24561 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4369c8d6ef1b413bb904bb0481ab3f77" candidate_uuid: "e8a08c18db504acb978da77ab5769df0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3297ca45214544db92252a24690e352e"
I20260809 15:39:46.006729 24416 raft_consensus.cc:3037] T 4369c8d6ef1b413bb904bb0481ab3f77 P 696cffb8c6ea4ad2aae2be03da3bb9c0 [term 0 FOLLOWER]: Advancing to term 1
I20260809 15:39:46.006824 24561 raft_consensus.cc:3037] T 4369c8d6ef1b413bb904bb0481ab3f77 P 3297ca45214544db92252a24690e352e [term 0 FOLLOWER]: Advancing to term 1
I20260809 15:39:46.007844 24561 raft_consensus.cc:2454] T 4369c8d6ef1b413bb904bb0481ab3f77 P 3297ca45214544db92252a24690e352e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e8a08c18db504acb978da77ab5769df0 in term 1.
I20260809 15:39:46.007895 24416 raft_consensus.cc:2454] T 4369c8d6ef1b413bb904bb0481ab3f77 P 696cffb8c6ea4ad2aae2be03da3bb9c0 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e8a08c18db504acb978da77ab5769df0 in term 1.
I20260809 15:39:46.008193 24201 leader_election.cc:302] T 4369c8d6ef1b413bb904bb0481ab3f77 P e8a08c18db504acb978da77ab5769df0 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3297ca45214544db92252a24690e352e, e8a08c18db504acb978da77ab5769df0; no voters:
I20260809 15:39:46.008437 24642 raft_consensus.cc:2781] T 4369c8d6ef1b413bb904bb0481ab3f77 P e8a08c18db504acb978da77ab5769df0 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 15:39:46.008560 24642 raft_consensus.cc:686] T 4369c8d6ef1b413bb904bb0481ab3f77 P e8a08c18db504acb978da77ab5769df0 [term 1 LEADER]: Becoming Leader. State: Replica: e8a08c18db504acb978da77ab5769df0, State: Running, Role: LEADER
I20260809 15:39:46.008889 24642 consensus_queue.cc:237] T 4369c8d6ef1b413bb904bb0481ab3f77 P e8a08c18db504acb978da77ab5769df0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "696cffb8c6ea4ad2aae2be03da3bb9c0" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 44749 } } peers { permanent_uuid: "3297ca45214544db92252a24690e352e" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 44119 } } peers { permanent_uuid: "e8a08c18db504acb978da77ab5769df0" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 34071 } }
I20260809 15:39:46.011976 24110 catalog_manager.cc:5495] T 4369c8d6ef1b413bb904bb0481ab3f77 P e8a08c18db504acb978da77ab5769df0 reported cstate change: term changed from 0 to 1, leader changed from <none> to e8a08c18db504acb978da77ab5769df0 (127.3.89.193). New cstate: current_term: 1 leader_uuid: "e8a08c18db504acb978da77ab5769df0" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "696cffb8c6ea4ad2aae2be03da3bb9c0" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 44749 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3297ca45214544db92252a24690e352e" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 44119 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e8a08c18db504acb978da77ab5769df0" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 34071 } health_report { overall_health: HEALTHY } } }
I20260809 15:39:46.033638 24633 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:39:46.029978 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47922 (local address 127.3.89.193:34071)
0809 15:39:46.030112 (+ 134us) server_negotiation.cc:206] Beginning negotiation
0809 15:39:46.030118 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 15:39:46.030135 (+ 17us) server_negotiation.cc:407] Connection header received
0809 15:39:46.030324 (+ 189us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 15:39:46.030326 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 15:39:46.030381 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 15:39:46.030514 (+ 133us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 15:39:46.030761 (+ 247us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:39:46.031333 (+ 572us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:39:46.032208 (+ 875us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:39:46.032427 (+ 219us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:39:46.032548 (+ 121us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:39:46.032846 (+ 298us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 15:39:46.033225 (+ 379us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 15:39:46.033366 (+ 141us) server_negotiation.cc:1035] Waiting for connection context
0809 15:39:46.033431 (+ 65us) server_negotiation.cc:299] Negotiation successful
0809 15:39:46.033494 (+ 63us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":48}
I20260809 15:39:46.039264 24416 raft_consensus.cc:1261] T 4369c8d6ef1b413bb904bb0481ab3f77 P 696cffb8c6ea4ad2aae2be03da3bb9c0 [term 1 FOLLOWER]: Refusing update from remote peer e8a08c18db504acb978da77ab5769df0: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 15:39:46.039263 24561 raft_consensus.cc:1261] T 4369c8d6ef1b413bb904bb0481ab3f77 P 3297ca45214544db92252a24690e352e [term 1 FOLLOWER]: Refusing update from remote peer e8a08c18db504acb978da77ab5769df0: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 15:39:46.039937 24642 consensus_queue.cc:1035] T 4369c8d6ef1b413bb904bb0481ab3f77 P e8a08c18db504acb978da77ab5769df0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3297ca45214544db92252a24690e352e" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 44119 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 15:39:46.040088 24643 consensus_queue.cc:1035] T 4369c8d6ef1b413bb904bb0481ab3f77 P e8a08c18db504acb978da77ab5769df0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "696cffb8c6ea4ad2aae2be03da3bb9c0" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 44749 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 15:39:46.047773 24651 mvcc.cc:205] Tried to move back new op lower bound from 7316643782809702400 to 7316643782695546880. Current Snapshot: MvccSnapshot[applied={T|T < 7316643782809702400}]
I20260809 15:39:46.049878 24650 mvcc.cc:205] Tried to move back new op lower bound from 7316643782809702400 to 7316643782695546880. Current Snapshot: MvccSnapshot[applied={T|T < 7316643782809702400 or (T in {7316643782809702400})}]
I20260809 15:39:46.057015 24111 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:52344:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 15:39:46.057179 24111 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:52344:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 15:39:46.059490 24111 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 99f9c73b4b6c4c54952df21cc345c7a6: Sending DeleteTablet for 3 replicas of tablet 4369c8d6ef1b413bb904bb0481ab3f77
I20260809 15:39:46.060168 24394 tablet_service.cc:1514] Processing DeleteTablet for tablet 4369c8d6ef1b413bb904bb0481ab3f77 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 15:39:46 UTC) from {username='kudu', principal='kudu/127.3.89.254@KRBTEST.COM'} at 127.0.0.1:39522
I20260809 15:39:46.060336 24538 tablet_service.cc:1514] Processing DeleteTablet for tablet 4369c8d6ef1b413bb904bb0481ab3f77 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 15:39:46 UTC) from {username='kudu', principal='kudu/127.3.89.254@KRBTEST.COM'} at 127.0.0.1:47256
I20260809 15:39:46.060464 24251 tablet_service.cc:1514] Processing DeleteTablet for tablet 4369c8d6ef1b413bb904bb0481ab3f77 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 15:39:46 UTC) from {username='kudu', principal='kudu/127.3.89.254@KRBTEST.COM'} at 127.0.0.1:47910
I20260809 15:39:46.060887 24663 tablet_replica.cc:330] T 4369c8d6ef1b413bb904bb0481ab3f77 P e8a08c18db504acb978da77ab5769df0: stopping tablet replica
I20260809 15:39:46.061080 24660 tablet_replica.cc:330] T 4369c8d6ef1b413bb904bb0481ab3f77 P 696cffb8c6ea4ad2aae2be03da3bb9c0: stopping tablet replica
I20260809 15:39:46.061173 24663 raft_consensus.cc:2229] T 4369c8d6ef1b413bb904bb0481ab3f77 P e8a08c18db504acb978da77ab5769df0 [term 1 LEADER]: Raft consensus shutting down.
I20260809 15:39:46.061450 3431 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskm314YR/build/debug/bin/kudu with pid 24182
I20260809 15:39:46.065074 24660 raft_consensus.cc:2229] T 4369c8d6ef1b413bb904bb0481ab3f77 P 696cffb8c6ea4ad2aae2be03da3bb9c0 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 15:39:46.065358 24660 raft_consensus.cc:2258] T 4369c8d6ef1b413bb904bb0481ab3f77 P 696cffb8c6ea4ad2aae2be03da3bb9c0 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 15:39:46.065725 24662 tablet_replica.cc:330] T 4369c8d6ef1b413bb904bb0481ab3f77 P 3297ca45214544db92252a24690e352e: stopping tablet replica
I20260809 15:39:46.066006 24662 raft_consensus.cc:2229] T 4369c8d6ef1b413bb904bb0481ab3f77 P 3297ca45214544db92252a24690e352e [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 15:39:46.066282 24662 raft_consensus.cc:2258] T 4369c8d6ef1b413bb904bb0481ab3f77 P 3297ca45214544db92252a24690e352e [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 15:39:46.066936 24660 ts_tablet_manager.cc:1905] T 4369c8d6ef1b413bb904bb0481ab3f77 P 696cffb8c6ea4ad2aae2be03da3bb9c0: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 15:39:46.067585 24662 ts_tablet_manager.cc:1905] T 4369c8d6ef1b413bb904bb0481ab3f77 P 3297ca45214544db92252a24690e352e: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 15:39:46.069233 24660 ts_tablet_manager.cc:1918] T 4369c8d6ef1b413bb904bb0481ab3f77 P 696cffb8c6ea4ad2aae2be03da3bb9c0: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 15:39:46.069332 24660 log.cc:1196] T 4369c8d6ef1b413bb904bb0481ab3f77 P 696cffb8c6ea4ad2aae2be03da3bb9c0: Deleting WAL directory at /tmp/dist-test-taskm314YR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786289732823074-3431-0/minicluster-data/ts-1/wal/wals/4369c8d6ef1b413bb904bb0481ab3f77
I20260809 15:39:46.069653 24660 ts_tablet_manager.cc:1939] T 4369c8d6ef1b413bb904bb0481ab3f77 P 696cffb8c6ea4ad2aae2be03da3bb9c0: Deleting consensus metadata
W20260809 15:39:46.070533 24094 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.3.89.193:34071 (error 108)
I20260809 15:39:46.070781 3431 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskm314YR/build/debug/bin/kudu with pid 24328
I20260809 15:39:46.070912 24662 ts_tablet_manager.cc:1918] T 4369c8d6ef1b413bb904bb0481ab3f77 P 3297ca45214544db92252a24690e352e: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 15:39:46.072058 24662 log.cc:1196] T 4369c8d6ef1b413bb904bb0481ab3f77 P 3297ca45214544db92252a24690e352e: Deleting WAL directory at /tmp/dist-test-taskm314YR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786289732823074-3431-0/minicluster-data/ts-2/wal/wals/4369c8d6ef1b413bb904bb0481ab3f77
I20260809 15:39:46.072578 24662 ts_tablet_manager.cc:1939] T 4369c8d6ef1b413bb904bb0481ab3f77 P 3297ca45214544db92252a24690e352e: Deleting consensus metadata
I20260809 15:39:46.074008 24094 catalog_manager.cc:4841] TS 3297ca45214544db92252a24690e352e (127.3.89.195:44119): tablet 4369c8d6ef1b413bb904bb0481ab3f77 (table test-table [id=bab709d3838b4542bbb6d19064adb2c0]) successfully deleted
I20260809 15:39:46.075762 24628 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:39:46.072337 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.89.193:34071 (local address 127.0.0.1:47932)
0809 15:39:46.075566 (+ 3229us) negotiation.cc:107] Waiting for socket to connect
0809 15:39:46.075648 (+ 82us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.89.193:34071: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":3124}
W20260809 15:39:46.075937 24094 catalog_manager.cc:4584] TS e8a08c18db504acb978da77ab5769df0 (127.3.89.193:34071): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 4369c8d6ef1b413bb904bb0481ab3f77: Network error: Client connection negotiation failed: client connection to 127.3.89.193:34071: connect: Connection refused (error 111)
I20260809 15:39:46.080230 3431 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskm314YR/build/debug/bin/kudu with pid 24473
I20260809 15:39:46.080701 24628 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:39:46.080388 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.89.194:44749 (local address 127.0.0.1:39524)
0809 15:39:46.080551 (+ 163us) negotiation.cc:107] Waiting for socket to connect
0809 15:39:46.080601 (+ 50us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.89.194:44749: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":84}
I20260809 15:39:46.086535 3431 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskm314YR/build/debug/bin/kudu with pid 24077
2026-08-09T15:39:46Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/4 (2284 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/5
2026-08-09T15:39:46Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T15:39:46Z Disabled control of system clock
I20260809 15:39:46.119081 3431 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskm314YR/build/debug/bin/kudu
/tmp/dist-test-taskm314YR/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786289732823074-3431-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786289732823074-3431-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786289732823074-3431-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786289732823074-3431-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.3.89.254:36375
--webserver_interface=127.3.89.254
--webserver_port=0
--builtin_ntp_servers=127.3.89.212:44477
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.3.89.254:36375
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260809 15:39:46.227872 24669 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 15:39:46.228127 24669 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 15:39:46.228160 24669 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 15:39:46.232028 24669 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 15:39:46.232076 24669 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 15:39:46.232092 24669 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 15:39:46.232108 24669 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 15:39:46.232123 24669 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 15:39:46.236748 24669 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.3.89.212:44477
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786289732823074-3431-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786289732823074-3431-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.3.89.254:36375
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.3.89.254:36375
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786289732823074-3431-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.3.89.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.24669
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786289732823074-3431-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 09 Aug 2026 14:46:55 UTC on ccycloud.quasar-yswkmf.root.comops.site
build id 4576540
I20260809 15:39:46.237664 24669 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 15:39:46.238945 24669 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 15:39:46.247139 24675 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 15:39:46.247155 24674 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 15:39:46.247169 24678 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 15:39:46.247350 24669 server_base.cc:1034] running on GCE node
I20260809 15:39:46.248008 24669 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 15:39:46.249025 24669 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 15:39:46.250219 24669 hybrid_clock.cc:648] HybridClock initialized: now 1786289986250203 us; error 30 us; skew 500 ppm
I20260809 15:39:46.252053 24669 webserver.cc:460] Webserver started at http://127.3.89.254:32851/ using document root <none> and password file <none>
I20260809 15:39:46.252596 24669 fs_manager.cc:359] Metadata directory not provided
I20260809 15:39:46.252689 24669 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 15:39:46.252924 24669 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 15:39:46.254946 24669 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskm314YR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786289732823074-3431-0/minicluster-data/master-0/data/instance:
uuid: "499b41fa1e164a888219f48bead27d72"
format_stamp: "Formatted at 2026-08-09 15:39:46 on dist-test-slave-2w3w"
server_key: "7c4d581f2418b4fad6d390251788da0f"
server_key_iv: "9a1144400830cbab0b857f3677d85342"
server_key_version: "encryptionkey@0"
I20260809 15:39:46.255569 24669 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskm314YR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786289732823074-3431-0/minicluster-data/master-0/wal/instance:
uuid: "499b41fa1e164a888219f48bead27d72"
format_stamp: "Formatted at 2026-08-09 15:39:46 on dist-test-slave-2w3w"
server_key: "7c4d581f2418b4fad6d390251788da0f"
server_key_iv: "9a1144400830cbab0b857f3677d85342"
server_key_version: "encryptionkey@0"
I20260809 15:39:46.259079 24669 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.001s sys 0.003s
I20260809 15:39:46.261296 24684 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 15:39:46.262665 24669 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260809 15:39:46.262859 24669 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskm314YR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786289732823074-3431-0/minicluster-data/master-0/data,/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786289732823074-3431-0/minicluster-data/master-0/wal
uuid: "499b41fa1e164a888219f48bead27d72"
format_stamp: "Formatted at 2026-08-09 15:39:46 on dist-test-slave-2w3w"
server_key: "7c4d581f2418b4fad6d390251788da0f"
server_key_iv: "9a1144400830cbab0b857f3677d85342"
server_key_version: "encryptionkey@0"
I20260809 15:39:46.263000 24669 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskm314YR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786289732823074-3431-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskm314YR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786289732823074-3431-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskm314YR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786289732823074-3431-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 15:39:46.300462 24669 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 15:39:46.301210 24669 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 15:39:46.301419 24669 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 15:39:46.308204 24669 rpc_server.cc:307] RPC server started. Bound to: 127.3.89.254:36375
I20260809 15:39:46.308240 24744 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.3.89.254:36375 every 8 connection(s)
I20260809 15:39:46.309110 24669 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskm314YR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786289732823074-3431-0/minicluster-data/master-0/data/info.pb
I20260809 15:39:46.312122 24745 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 15:39:46.314801 3431 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskm314YR/build/debug/bin/kudu as pid 24669
I20260809 15:39:46.314983 3431 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskm314YR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786289732823074-3431-0/minicluster-data/master-0/wal/instance
I20260809 15:39:46.315269 3431 external_mini_cluster.cc:1428] Setting key 566772350e329ed0fcf9ba0f3da2f025
I20260809 15:39:46.317984 24745 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 499b41fa1e164a888219f48bead27d72: Bootstrap starting.
I20260809 15:39:46.320420 24745 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 499b41fa1e164a888219f48bead27d72: Neither blocks nor log segments found. Creating new log.
I20260809 15:39:46.321285 24745 log.cc:824] T 00000000000000000000000000000000 P 499b41fa1e164a888219f48bead27d72: Log is configured to *not* fsync() on all Append() calls
I20260809 15:39:46.323400 24745 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 499b41fa1e164a888219f48bead27d72: No bootstrap required, opened a new log
I20260809 15:39:46.324472 24748 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:39:46.316557 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34360 (local address 127.3.89.254:36375)
0809 15:39:46.317224 (+ 667us) server_negotiation.cc:206] Beginning negotiation
0809 15:39:46.317240 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0809 15:39:46.317278 (+ 38us) server_negotiation.cc:407] Connection header received
0809 15:39:46.317885 (+ 607us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 15:39:46.317900 (+ 15us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 15:39:46.318313 (+ 413us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 15:39:46.318575 (+ 262us) server_negotiation.cc:226] Negotiated authn=SASL
0809 15:39:46.319004 (+ 429us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:39:46.319951 (+ 947us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:39:46.321217 (+ 1266us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:39:46.321789 (+ 572us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:39:46.321981 (+ 192us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:39:46.322766 (+ 785us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 15:39:46.322771 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 15:39:46.322781 (+ 10us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 15:39:46.323207 (+ 426us) server_negotiation.cc:922] calling sasl_server_start()
0809 15:39:46.323263 (+ 56us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 15:39:46.323283 (+ 20us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 15:39:46.323359 (+ 76us) server_negotiation.cc:1035] Waiting for connection context
0809 15:39:46.323554 (+ 195us) server_negotiation.cc:299] Negotiation successful
0809 15:39:46.323835 (+ 281us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":266,"thread_start_us":138,"threads_started":1}
I20260809 15:39:46.326740 24745 raft_consensus.cc:348] T 00000000000000000000000000000000 P 499b41fa1e164a888219f48bead27d72 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "499b41fa1e164a888219f48bead27d72" member_type: VOTER last_known_addr { host: "127.3.89.254" port: 36375 } }
I20260809 15:39:46.327056 24745 raft_consensus.cc:374] T 00000000000000000000000000000000 P 499b41fa1e164a888219f48bead27d72 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 15:39:46.327138 24745 raft_consensus.cc:729] T 00000000000000000000000000000000 P 499b41fa1e164a888219f48bead27d72 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 499b41fa1e164a888219f48bead27d72, State: Initialized, Role: FOLLOWER
I20260809 15:39:46.327469 24745 consensus_queue.cc:260] T 00000000000000000000000000000000 P 499b41fa1e164a888219f48bead27d72 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "499b41fa1e164a888219f48bead27d72" member_type: VOTER last_known_addr { host: "127.3.89.254" port: 36375 } }
I20260809 15:39:46.327576 24745 raft_consensus.cc:388] T 00000000000000000000000000000000 P 499b41fa1e164a888219f48bead27d72 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 15:39:46.327620 24745 raft_consensus.cc:482] T 00000000000000000000000000000000 P 499b41fa1e164a888219f48bead27d72 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 15:39:46.327687 24745 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 499b41fa1e164a888219f48bead27d72 [term 0 FOLLOWER]: Advancing to term 1
I20260809 15:39:46.328437 24745 raft_consensus.cc:504] T 00000000000000000000000000000000 P 499b41fa1e164a888219f48bead27d72 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "499b41fa1e164a888219f48bead27d72" member_type: VOTER last_known_addr { host: "127.3.89.254" port: 36375 } }
I20260809 15:39:46.328701 24745 leader_election.cc:302] T 00000000000000000000000000000000 P 499b41fa1e164a888219f48bead27d72 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 499b41fa1e164a888219f48bead27d72; no voters:
I20260809 15:39:46.328927 24745 leader_election.cc:288] T 00000000000000000000000000000000 P 499b41fa1e164a888219f48bead27d72 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 15:39:46.329049 24750 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 499b41fa1e164a888219f48bead27d72 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 15:39:46.329268 24750 raft_consensus.cc:686] T 00000000000000000000000000000000 P 499b41fa1e164a888219f48bead27d72 [term 1 LEADER]: Becoming Leader. State: Replica: 499b41fa1e164a888219f48bead27d72, State: Running, Role: LEADER
I20260809 15:39:46.329574 24750 consensus_queue.cc:237] T 00000000000000000000000000000000 P 499b41fa1e164a888219f48bead27d72 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "499b41fa1e164a888219f48bead27d72" member_type: VOTER last_known_addr { host: "127.3.89.254" port: 36375 } }
I20260809 15:39:46.330153 24745 sys_catalog.cc:564] T 00000000000000000000000000000000 P 499b41fa1e164a888219f48bead27d72 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 15:39:46.331156 24751 sys_catalog.cc:455] T 00000000000000000000000000000000 P 499b41fa1e164a888219f48bead27d72 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "499b41fa1e164a888219f48bead27d72" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "499b41fa1e164a888219f48bead27d72" member_type: VOTER last_known_addr { host: "127.3.89.254" port: 36375 } } }
I20260809 15:39:46.331288 24751 sys_catalog.cc:458] T 00000000000000000000000000000000 P 499b41fa1e164a888219f48bead27d72 [sys.catalog]: This master's current role is: LEADER
I20260809 15:39:46.331658 24760 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 15:39:46.332347 24752 sys_catalog.cc:455] T 00000000000000000000000000000000 P 499b41fa1e164a888219f48bead27d72 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 499b41fa1e164a888219f48bead27d72. Latest consensus state: current_term: 1 leader_uuid: "499b41fa1e164a888219f48bead27d72" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "499b41fa1e164a888219f48bead27d72" member_type: VOTER last_known_addr { host: "127.3.89.254" port: 36375 } } }
I20260809 15:39:46.332453 24752 sys_catalog.cc:458] T 00000000000000000000000000000000 P 499b41fa1e164a888219f48bead27d72 [sys.catalog]: This master's current role is: LEADER
I20260809 15:39:46.334640 24760 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 15:39:46.339743 24760 catalog_manager.cc:1347] Generated new cluster ID: 3bfa58aeff5248a6bec40b348d44c8e3
I20260809 15:39:46.339816 24760 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 15:39:46.348016 24760 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 15:39:46.348790 24760 catalog_manager.cc:1504] Loading token signing keys...
I20260809 15:39:46.355868 24760 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 499b41fa1e164a888219f48bead27d72: Generated new TSK 0
I20260809 15:39:46.356478 24760 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260809 15:39:46.366704 3431 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskm314YR/build/debug/bin/kudu
/tmp/dist-test-taskm314YR/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786289732823074-3431-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786289732823074-3431-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786289732823074-3431-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786289732823074-3431-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.3.89.193:0
--local_ip_for_outbound_sockets=127.3.89.193
--webserver_interface=127.3.89.193
--webserver_port=0
--tserver_master_addrs=127.3.89.254:36375
--builtin_ntp_servers=127.3.89.212:44477
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260809 15:39:46.478581 24772 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 15:39:46.478868 24772 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 15:39:46.478941 24772 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 15:39:46.483023 24772 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 15:39:46.483080 24772 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 15:39:46.483145 24772 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.3.89.193
I20260809 15:39:46.487830 24772 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.3.89.212:44477
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786289732823074-3431-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786289732823074-3431-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.3.89.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786289732823074-3431-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.3.89.193
--webserver_port=0
--tserver_master_addrs=127.3.89.254:36375
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.24772
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.3.89.193
--log_dir=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786289732823074-3431-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 09 Aug 2026 14:46:55 UTC on ccycloud.quasar-yswkmf.root.comops.site
build id 4576540
I20260809 15:39:46.488849 24772 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 15:39:46.490178 24772 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 15:39:46.499655 24777 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 15:39:46.499704 24772 server_base.cc:1034] running on GCE node
W20260809 15:39:46.499661 24778 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 15:39:46.499655 24780 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 15:39:46.500368 24772 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 15:39:46.500986 24772 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 15:39:46.502183 24772 hybrid_clock.cc:648] HybridClock initialized: now 1786289986502168 us; error 27 us; skew 500 ppm
I20260809 15:39:46.504101 24772 webserver.cc:460] Webserver started at http://127.3.89.193:44415/ using document root <none> and password file <none>
I20260809 15:39:46.504669 24772 fs_manager.cc:359] Metadata directory not provided
I20260809 15:39:46.504769 24772 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 15:39:46.504990 24772 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 15:39:46.506949 24772 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskm314YR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786289732823074-3431-0/minicluster-data/ts-0/data/instance:
uuid: "5109319448fd46689c9f6b6c07ad9af4"
format_stamp: "Formatted at 2026-08-09 15:39:46 on dist-test-slave-2w3w"
server_key: "41121f4c609efd2d7c15cbb09182829a"
server_key_iv: "b1b12d438eba76941abcf73ca52ce37c"
server_key_version: "encryptionkey@0"
I20260809 15:39:46.507601 24772 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskm314YR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786289732823074-3431-0/minicluster-data/ts-0/wal/instance:
uuid: "5109319448fd46689c9f6b6c07ad9af4"
format_stamp: "Formatted at 2026-08-09 15:39:46 on dist-test-slave-2w3w"
server_key: "41121f4c609efd2d7c15cbb09182829a"
server_key_iv: "b1b12d438eba76941abcf73ca52ce37c"
server_key_version: "encryptionkey@0"
I20260809 15:39:46.511166 24772 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.001s sys 0.003s
I20260809 15:39:46.513471 24786 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 15:39:46.514662 24772 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260809 15:39:46.514765 24772 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskm314YR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786289732823074-3431-0/minicluster-data/ts-0/data,/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786289732823074-3431-0/minicluster-data/ts-0/wal
uuid: "5109319448fd46689c9f6b6c07ad9af4"
format_stamp: "Formatted at 2026-08-09 15:39:46 on dist-test-slave-2w3w"
server_key: "41121f4c609efd2d7c15cbb09182829a"
server_key_iv: "b1b12d438eba76941abcf73ca52ce37c"
server_key_version: "encryptionkey@0"
I20260809 15:39:46.514846 24772 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskm314YR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786289732823074-3431-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskm314YR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786289732823074-3431-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskm314YR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786289732823074-3431-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 15:39:46.532431 24772 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 15:39:46.532976 24772 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 15:39:46.533113 24772 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 15:39:46.533605 24772 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 15:39:46.534468 24772 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 15:39:46.534515 24772 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 15:39:46.534552 24772 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 15:39:46.534567 24772 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 15:39:46.542963 24772 rpc_server.cc:307] RPC server started. Bound to: 127.3.89.193:36399
I20260809 15:39:46.543045 24913 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.3.89.193:36399 every 8 connection(s)
I20260809 15:39:46.543826 24772 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskm314YR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786289732823074-3431-0/minicluster-data/ts-0/data/info.pb
I20260809 15:39:46.552299 24748 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:39:46.545511 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.89.193:45139 (local address 127.3.89.254:36375)
0809 15:39:46.545651 (+ 140us) server_negotiation.cc:206] Beginning negotiation
0809 15:39:46.545655 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 15:39:46.546638 (+ 983us) server_negotiation.cc:407] Connection header received
0809 15:39:46.548018 (+ 1380us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 15:39:46.548021 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 15:39:46.548066 (+ 45us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 15:39:46.548142 (+ 76us) server_negotiation.cc:226] Negotiated authn=SASL
0809 15:39:46.548971 (+ 829us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:39:46.549500 (+ 529us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:39:46.550567 (+ 1067us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:39:46.550776 (+ 209us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:39:46.550852 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:39:46.551892 (+ 1040us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 15:39:46.551896 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 15:39:46.551898 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 15:39:46.551910 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0809 15:39:46.551929 (+ 19us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 15:39:46.551938 (+ 9us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 15:39:46.552006 (+ 68us) server_negotiation.cc:1035] Waiting for connection context
0809 15:39:46.552138 (+ 132us) server_negotiation.cc:299] Negotiation successful
0809 15:39:46.552190 (+ 52us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":57}
I20260809 15:39:46.553087 24916 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:39:46.545880 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.89.254:36375 (local address 127.3.89.193:45139)
0809 15:39:46.546500 (+ 620us) negotiation.cc:107] Waiting for socket to connect
0809 15:39:46.546538 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0809 15:39:46.547826 (+ 1288us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 15:39:46.548310 (+ 484us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 15:39:46.548316 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 15:39:46.548395 (+ 79us) client_negotiation.cc:189] Negotiated authn=SASL
0809 15:39:46.548841 (+ 446us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:39:46.548854 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:39:46.549616 (+ 762us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:39:46.549618 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:39:46.550461 (+ 843us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:39:46.550470 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:39:46.550905 (+ 435us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:39:46.550907 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:39:46.551100 (+ 193us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:39:46.551650 (+ 550us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0809 15:39:46.551652 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0809 15:39:46.551688 (+ 36us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0809 15:39:46.551746 (+ 58us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 15:39:46.552020 (+ 274us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 15:39:46.552027 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 15:39:46.552031 (+ 4us) client_negotiation.cc:769] Sending connection context
0809 15:39:46.552119 (+ 88us) client_negotiation.cc:240] Negotiation successful
0809 15:39:46.552333 (+ 214us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":256,"thread_start_us":111,"threads_started":1}
I20260809 15:39:46.553995 3431 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskm314YR/build/debug/bin/kudu as pid 24772
I20260809 15:39:46.554131 3431 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskm314YR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786289732823074-3431-0/minicluster-data/ts-0/wal/instance
I20260809 15:39:46.554386 24914 heartbeater.cc:347] Connected to a master server at 127.3.89.254:36375
I20260809 15:39:46.554426 3431 external_mini_cluster.cc:1428] Setting key 6b3835664ab4d707563fe19abba8a8b0
I20260809 15:39:46.554623 24914 heartbeater.cc:464] Registering TS with master...
I20260809 15:39:46.555244 24914 heartbeater.cc:511] Master 127.3.89.254:36375 requested a full tablet report, sending...
I20260809 15:39:46.556969 3431 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskm314YR/build/debug/bin/kudu
/tmp/dist-test-taskm314YR/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786289732823074-3431-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786289732823074-3431-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786289732823074-3431-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786289732823074-3431-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.3.89.194:0
--local_ip_for_outbound_sockets=127.3.89.194
--webserver_interface=127.3.89.194
--webserver_port=0
--tserver_master_addrs=127.3.89.254:36375
--builtin_ntp_servers=127.3.89.212:44477
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260809 15:39:46.557107 24703 ts_manager.cc:194] Registered new tserver with Master: 5109319448fd46689c9f6b6c07ad9af4 (127.3.89.193:36399)
I20260809 15:39:46.558466 24703 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.3.89.193:45139
W20260809 15:39:46.666772 24917 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 15:39:46.667006 24917 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 15:39:46.667042 24917 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 15:39:46.670975 24917 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 15:39:46.671021 24917 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 15:39:46.671073 24917 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.3.89.194
I20260809 15:39:46.675567 24917 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.3.89.212:44477
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786289732823074-3431-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786289732823074-3431-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.3.89.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786289732823074-3431-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.3.89.194
--webserver_port=0
--tserver_master_addrs=127.3.89.254:36375
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.24917
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.3.89.194
--log_dir=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786289732823074-3431-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 09 Aug 2026 14:46:55 UTC on ccycloud.quasar-yswkmf.root.comops.site
build id 4576540
I20260809 15:39:46.676466 24917 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 15:39:46.677702 24917 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 15:39:46.686792 24926 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 15:39:46.686775 24924 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 15:39:46.686754 24923 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 15:39:46.686829 24917 server_base.cc:1034] running on GCE node
I20260809 15:39:46.687302 24917 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 15:39:46.687917 24917 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 15:39:46.689118 24917 hybrid_clock.cc:648] HybridClock initialized: now 1786289986689105 us; error 25 us; skew 500 ppm
I20260809 15:39:46.690886 24917 webserver.cc:460] Webserver started at http://127.3.89.194:44407/ using document root <none> and password file <none>
I20260809 15:39:46.691461 24917 fs_manager.cc:359] Metadata directory not provided
I20260809 15:39:46.691555 24917 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 15:39:46.691773 24917 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 15:39:46.693728 24917 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskm314YR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786289732823074-3431-0/minicluster-data/ts-1/data/instance:
uuid: "01481569ef2c4a0c9c5e72569fd73b95"
format_stamp: "Formatted at 2026-08-09 15:39:46 on dist-test-slave-2w3w"
server_key: "f0529aac9a3de84759382038de61f64b"
server_key_iv: "7fec28bf08c6e2e7233e3e49ba66d9dc"
server_key_version: "encryptionkey@0"
I20260809 15:39:46.694281 24917 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskm314YR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786289732823074-3431-0/minicluster-data/ts-1/wal/instance:
uuid: "01481569ef2c4a0c9c5e72569fd73b95"
format_stamp: "Formatted at 2026-08-09 15:39:46 on dist-test-slave-2w3w"
server_key: "f0529aac9a3de84759382038de61f64b"
server_key_iv: "7fec28bf08c6e2e7233e3e49ba66d9dc"
server_key_version: "encryptionkey@0"
I20260809 15:39:46.697749 24917 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.006s sys 0.000s
I20260809 15:39:46.700006 24932 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 15:39:46.701115 24917 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260809 15:39:46.701248 24917 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskm314YR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786289732823074-3431-0/minicluster-data/ts-1/data,/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786289732823074-3431-0/minicluster-data/ts-1/wal
uuid: "01481569ef2c4a0c9c5e72569fd73b95"
format_stamp: "Formatted at 2026-08-09 15:39:46 on dist-test-slave-2w3w"
server_key: "f0529aac9a3de84759382038de61f64b"
server_key_iv: "7fec28bf08c6e2e7233e3e49ba66d9dc"
server_key_version: "encryptionkey@0"
I20260809 15:39:46.701359 24917 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskm314YR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786289732823074-3431-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskm314YR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786289732823074-3431-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskm314YR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786289732823074-3431-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 15:39:46.733968 24917 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 15:39:46.734670 24917 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 15:39:46.734884 24917 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 15:39:46.735509 24917 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 15:39:46.736466 24917 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 15:39:46.736541 24917 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 15:39:46.736617 24917 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 15:39:46.736660 24917 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 15:39:46.745179 24917 rpc_server.cc:307] RPC server started. Bound to: 127.3.89.194:38127
I20260809 15:39:46.745220 25059 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.3.89.194:38127 every 8 connection(s)
I20260809 15:39:46.746070 24917 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskm314YR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786289732823074-3431-0/minicluster-data/ts-1/data/info.pb
I20260809 15:39:46.754705 3431 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskm314YR/build/debug/bin/kudu as pid 24917
I20260809 15:39:46.754801 24748 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:39:46.747829 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.89.194:46677 (local address 127.3.89.254:36375)
0809 15:39:46.747979 (+ 150us) server_negotiation.cc:206] Beginning negotiation
0809 15:39:46.747984 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 15:39:46.749039 (+ 1055us) server_negotiation.cc:407] Connection header received
0809 15:39:46.750344 (+ 1305us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 15:39:46.750347 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 15:39:46.750390 (+ 43us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 15:39:46.750465 (+ 75us) server_negotiation.cc:226] Negotiated authn=SASL
0809 15:39:46.751308 (+ 843us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:39:46.751836 (+ 528us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:39:46.752910 (+ 1074us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:39:46.753120 (+ 210us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:39:46.753199 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:39:46.754234 (+ 1035us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 15:39:46.754237 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 15:39:46.754239 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 15:39:46.754250 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0809 15:39:46.754273 (+ 23us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 15:39:46.754299 (+ 26us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 15:39:46.754363 (+ 64us) server_negotiation.cc:1035] Waiting for connection context
0809 15:39:46.754536 (+ 173us) server_negotiation.cc:299] Negotiation successful
0809 15:39:46.754605 (+ 69us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":55}
I20260809 15:39:46.754901 3431 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskm314YR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786289732823074-3431-0/minicluster-data/ts-1/wal/instance
I20260809 15:39:46.755261 3431 external_mini_cluster.cc:1428] Setting key da78b086b017c26d73120a12f44bdc61
I20260809 15:39:46.755780 25062 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:39:46.748266 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.89.254:36375 (local address 127.3.89.194:46677)
0809 15:39:46.748883 (+ 617us) negotiation.cc:107] Waiting for socket to connect
0809 15:39:46.748923 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0809 15:39:46.750156 (+ 1233us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 15:39:46.750584 (+ 428us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 15:39:46.750590 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 15:39:46.750665 (+ 75us) client_negotiation.cc:189] Negotiated authn=SASL
0809 15:39:46.751162 (+ 497us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:39:46.751174 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:39:46.751944 (+ 770us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:39:46.751947 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:39:46.752797 (+ 850us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:39:46.752805 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:39:46.753245 (+ 440us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:39:46.753248 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:39:46.753381 (+ 133us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:39:46.754000 (+ 619us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0809 15:39:46.754001 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0809 15:39:46.754039 (+ 38us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0809 15:39:46.754097 (+ 58us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 15:39:46.754378 (+ 281us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 15:39:46.754384 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 15:39:46.754388 (+ 4us) client_negotiation.cc:769] Sending connection context
0809 15:39:46.754481 (+ 93us) client_negotiation.cc:240] Negotiation successful
0809 15:39:46.754685 (+ 204us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":255,"thread_start_us":114,"threads_started":1}
I20260809 15:39:46.756906 25060 heartbeater.cc:347] Connected to a master server at 127.3.89.254:36375
I20260809 15:39:46.757153 25060 heartbeater.cc:464] Registering TS with master...
I20260809 15:39:46.757421 3431 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskm314YR/build/debug/bin/kudu
/tmp/dist-test-taskm314YR/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786289732823074-3431-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786289732823074-3431-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786289732823074-3431-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786289732823074-3431-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.3.89.195:0
--local_ip_for_outbound_sockets=127.3.89.195
--webserver_interface=127.3.89.195
--webserver_port=0
--tserver_master_addrs=127.3.89.254:36375
--builtin_ntp_servers=127.3.89.212:44477
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260809 15:39:46.757913 25060 heartbeater.cc:511] Master 127.3.89.254:36375 requested a full tablet report, sending...
I20260809 15:39:46.758949 24703 ts_manager.cc:194] Registered new tserver with Master: 01481569ef2c4a0c9c5e72569fd73b95 (127.3.89.194:38127)
I20260809 15:39:46.759615 24703 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.3.89.194:46677
W20260809 15:39:46.866138 25063 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 15:39:46.866380 25063 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 15:39:46.866420 25063 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 15:39:46.870486 25063 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 15:39:46.870540 25063 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 15:39:46.870621 25063 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.3.89.195
I20260809 15:39:46.875348 25063 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.3.89.212:44477
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786289732823074-3431-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786289732823074-3431-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.3.89.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786289732823074-3431-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.3.89.195
--webserver_port=0
--tserver_master_addrs=127.3.89.254:36375
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.25063
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.3.89.195
--log_dir=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786289732823074-3431-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 09 Aug 2026 14:46:55 UTC on ccycloud.quasar-yswkmf.root.comops.site
build id 4576540
I20260809 15:39:46.876368 25063 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 15:39:46.877724 25063 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 15:39:46.886945 25069 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 15:39:46.886986 25068 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 15:39:46.886992 25071 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 15:39:46.887157 25063 server_base.cc:1034] running on GCE node
I20260809 15:39:46.887593 25063 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 15:39:46.888146 25063 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 15:39:46.889312 25063 hybrid_clock.cc:648] HybridClock initialized: now 1786289986889286 us; error 37 us; skew 500 ppm
I20260809 15:39:46.891144 25063 webserver.cc:460] Webserver started at http://127.3.89.195:44483/ using document root <none> and password file <none>
I20260809 15:39:46.891683 25063 fs_manager.cc:359] Metadata directory not provided
I20260809 15:39:46.891757 25063 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 15:39:46.891957 25063 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 15:39:46.894052 25063 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskm314YR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786289732823074-3431-0/minicluster-data/ts-2/data/instance:
uuid: "2f5c45c1f3c449d7a50861b9f201bb25"
format_stamp: "Formatted at 2026-08-09 15:39:46 on dist-test-slave-2w3w"
server_key: "84f9564d170d33b831aa41ff0af1a305"
server_key_iv: "ab1c15180ac0eb36a1d2c193deb57709"
server_key_version: "encryptionkey@0"
I20260809 15:39:46.894603 25063 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskm314YR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786289732823074-3431-0/minicluster-data/ts-2/wal/instance:
uuid: "2f5c45c1f3c449d7a50861b9f201bb25"
format_stamp: "Formatted at 2026-08-09 15:39:46 on dist-test-slave-2w3w"
server_key: "84f9564d170d33b831aa41ff0af1a305"
server_key_iv: "ab1c15180ac0eb36a1d2c193deb57709"
server_key_version: "encryptionkey@0"
I20260809 15:39:46.898070 25063 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.003s sys 0.003s
I20260809 15:39:46.900326 25077 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 15:39:46.901437 25063 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260809 15:39:46.901547 25063 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskm314YR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786289732823074-3431-0/minicluster-data/ts-2/data,/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786289732823074-3431-0/minicluster-data/ts-2/wal
uuid: "2f5c45c1f3c449d7a50861b9f201bb25"
format_stamp: "Formatted at 2026-08-09 15:39:46 on dist-test-slave-2w3w"
server_key: "84f9564d170d33b831aa41ff0af1a305"
server_key_iv: "ab1c15180ac0eb36a1d2c193deb57709"
server_key_version: "encryptionkey@0"
I20260809 15:39:46.901660 25063 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskm314YR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786289732823074-3431-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskm314YR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786289732823074-3431-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskm314YR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786289732823074-3431-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 15:39:46.915376 25063 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 15:39:46.915964 25063 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 15:39:46.916141 25063 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 15:39:46.916669 25063 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 15:39:46.917524 25063 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 15:39:46.917573 25063 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 15:39:46.917639 25063 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 15:39:46.917673 25063 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 15:39:46.926052 25063 rpc_server.cc:307] RPC server started. Bound to: 127.3.89.195:41135
I20260809 15:39:46.926079 25206 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.3.89.195:41135 every 8 connection(s)
I20260809 15:39:46.926851 25063 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskm314YR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786289732823074-3431-0/minicluster-data/ts-2/data/info.pb
I20260809 15:39:46.934183 3431 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskm314YR/build/debug/bin/kudu as pid 25063
I20260809 15:39:46.934305 3431 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskm314YR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786289732823074-3431-0/minicluster-data/ts-2/wal/instance
I20260809 15:39:46.934582 3431 external_mini_cluster.cc:1428] Setting key aed37c673d2719921b806bd520db892f
I20260809 15:39:46.935904 24748 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:39:46.928547 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.89.195:37435 (local address 127.3.89.254:36375)
0809 15:39:46.928681 (+ 134us) server_negotiation.cc:206] Beginning negotiation
0809 15:39:46.928685 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 15:39:46.929625 (+ 940us) server_negotiation.cc:407] Connection header received
0809 15:39:46.931122 (+ 1497us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 15:39:46.931126 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 15:39:46.931184 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 15:39:46.931260 (+ 76us) server_negotiation.cc:226] Negotiated authn=SASL
0809 15:39:46.932148 (+ 888us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:39:46.932686 (+ 538us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:39:46.933760 (+ 1074us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:39:46.934063 (+ 303us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:39:46.934183 (+ 120us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:39:46.935409 (+ 1226us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 15:39:46.935413 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 15:39:46.935416 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 15:39:46.935435 (+ 19us) server_negotiation.cc:922] calling sasl_server_start()
0809 15:39:46.935454 (+ 19us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 15:39:46.935463 (+ 9us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 15:39:46.935544 (+ 81us) server_negotiation.cc:1035] Waiting for connection context
0809 15:39:46.935710 (+ 166us) server_negotiation.cc:299] Negotiation successful
0809 15:39:46.935769 (+ 59us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":48}
I20260809 15:39:46.936753 25209 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:39:46.928901 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.89.254:36375 (local address 127.3.89.195:37435)
0809 15:39:46.929505 (+ 604us) negotiation.cc:107] Waiting for socket to connect
0809 15:39:46.929541 (+ 36us) client_negotiation.cc:174] Beginning negotiation
0809 15:39:46.930904 (+ 1363us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 15:39:46.931411 (+ 507us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 15:39:46.931416 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 15:39:46.931504 (+ 88us) client_negotiation.cc:189] Negotiated authn=SASL
0809 15:39:46.932015 (+ 511us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:39:46.932041 (+ 26us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:39:46.932809 (+ 768us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:39:46.932811 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:39:46.933653 (+ 842us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:39:46.933661 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:39:46.934207 (+ 546us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:39:46.934209 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:39:46.934433 (+ 224us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:39:46.935117 (+ 684us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0809 15:39:46.935119 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0809 15:39:46.935168 (+ 49us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0809 15:39:46.935240 (+ 72us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 15:39:46.935546 (+ 306us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 15:39:46.935557 (+ 11us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 15:39:46.935566 (+ 9us) client_negotiation.cc:769] Sending connection context
0809 15:39:46.935704 (+ 138us) client_negotiation.cc:240] Negotiation successful
0809 15:39:46.935954 (+ 250us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":256,"thread_start_us":108,"threads_started":1}
I20260809 15:39:46.937829 25207 heartbeater.cc:347] Connected to a master server at 127.3.89.254:36375
I20260809 15:39:46.938024 25207 heartbeater.cc:464] Registering TS with master...
I20260809 15:39:46.938433 25207 heartbeater.cc:511] Master 127.3.89.254:36375 requested a full tablet report, sending...
I20260809 15:39:46.939380 24703 ts_manager.cc:194] Registered new tserver with Master: 2f5c45c1f3c449d7a50861b9f201bb25 (127.3.89.195:41135)
I20260809 15:39:46.939982 24703 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.3.89.195:37435
I20260809 15:39:46.948621 3431 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260809 15:39:46.954162 24748 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:39:46.950624 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34364 (local address 127.3.89.254:36375)
0809 15:39:46.950782 (+ 158us) server_negotiation.cc:206] Beginning negotiation
0809 15:39:46.950787 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 15:39:46.950895 (+ 108us) server_negotiation.cc:407] Connection header received
0809 15:39:46.951033 (+ 138us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 15:39:46.951035 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 15:39:46.951075 (+ 40us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 15:39:46.951140 (+ 65us) server_negotiation.cc:226] Negotiated authn=SASL
0809 15:39:46.951436 (+ 296us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:39:46.951945 (+ 509us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:39:46.952755 (+ 810us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:39:46.952966 (+ 211us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:39:46.953041 (+ 75us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:39:46.953753 (+ 712us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 15:39:46.953756 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 15:39:46.953758 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 15:39:46.953769 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0809 15:39:46.953787 (+ 18us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 15:39:46.953795 (+ 8us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 15:39:46.953854 (+ 59us) server_negotiation.cc:1035] Waiting for connection context
0809 15:39:46.953929 (+ 75us) server_negotiation.cc:299] Negotiation successful
0809 15:39:46.953984 (+ 55us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":71}
I20260809 15:39:46.956748 24703 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:34364:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260809 15:39:46.959754 24703 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260809 15:39:46.974156 25223 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:39:46.969444 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50414 (local address 127.3.89.194:38127)
0809 15:39:46.969840 (+ 396us) server_negotiation.cc:206] Beginning negotiation
0809 15:39:46.969854 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0809 15:39:46.969879 (+ 25us) server_negotiation.cc:407] Connection header received
0809 15:39:46.969951 (+ 72us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 15:39:46.969954 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 15:39:46.970074 (+ 120us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 15:39:46.970223 (+ 149us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 15:39:46.970463 (+ 240us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:39:46.971424 (+ 961us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:39:46.972946 (+ 1522us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:39:46.973486 (+ 540us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:39:46.973615 (+ 129us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:39:46.973816 (+ 201us) server_negotiation.cc:1035] Waiting for connection context
0809 15:39:46.973911 (+ 95us) server_negotiation.cc:299] Negotiation successful
0809 15:39:46.974014 (+ 103us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":272,"thread_start_us":148,"threads_started":1}
I20260809 15:39:46.974151 25218 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:39:46.968796 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.89.194:38127 (local address 127.0.0.1:50414)
0809 15:39:46.969402 (+ 606us) negotiation.cc:107] Waiting for socket to connect
0809 15:39:46.969415 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0809 15:39:46.969485 (+ 70us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 15:39:46.970202 (+ 717us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 15:39:46.970205 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 15:39:46.970222 (+ 17us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 15:39:46.970329 (+ 107us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:39:46.970339 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:39:46.971561 (+ 1222us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:39:46.971566 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:39:46.972823 (+ 1257us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:39:46.972832 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:39:46.973616 (+ 784us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:39:46.973619 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:39:46.973723 (+ 104us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:39:46.973760 (+ 37us) client_negotiation.cc:769] Sending connection context
0809 15:39:46.973871 (+ 111us) client_negotiation.cc:240] Negotiation successful
0809 15:39:46.973990 (+ 119us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":550,"mutex_wait_us":22,"thread_start_us":95,"threads_started":1}
I20260809 15:39:46.974572 25219 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:39:46.968100 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.89.193:36399 (local address 127.0.0.1:49930)
0809 15:39:46.968581 (+ 481us) negotiation.cc:107] Waiting for socket to connect
0809 15:39:46.968630 (+ 49us) client_negotiation.cc:174] Beginning negotiation
0809 15:39:46.968761 (+ 131us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 15:39:46.969566 (+ 805us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 15:39:46.969572 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 15:39:46.969590 (+ 18us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 15:39:46.969767 (+ 177us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:39:46.969774 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:39:46.971226 (+ 1452us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:39:46.971229 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:39:46.972709 (+ 1480us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:39:46.972719 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:39:46.973998 (+ 1279us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:39:46.974002 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:39:46.974237 (+ 235us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:39:46.974268 (+ 31us) client_negotiation.cc:769] Sending connection context
0809 15:39:46.974353 (+ 85us) client_negotiation.cc:240] Negotiation successful
0809 15:39:46.974424 (+ 71us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":320,"mutex_wait_us":131,"thread_start_us":66,"threads_started":1}
I20260809 15:39:46.974800 25221 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:39:46.968186 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.89.195:41135 (local address 127.0.0.1:49030)
0809 15:39:46.969087 (+ 901us) negotiation.cc:107] Waiting for socket to connect
0809 15:39:46.969102 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0809 15:39:46.969199 (+ 97us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 15:39:46.970371 (+ 1172us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 15:39:46.970376 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 15:39:46.970398 (+ 22us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 15:39:46.970485 (+ 87us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:39:46.970490 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:39:46.971722 (+ 1232us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:39:46.971729 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:39:46.972735 (+ 1006us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:39:46.972744 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:39:46.973998 (+ 1254us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:39:46.974002 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:39:46.974388 (+ 386us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:39:46.974408 (+ 20us) client_negotiation.cc:769] Sending connection context
0809 15:39:46.974474 (+ 66us) client_negotiation.cc:240] Negotiation successful
0809 15:39:46.974534 (+ 60us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":838,"mutex_wait_us":72,"thread_start_us":115,"threads_started":1}
I20260809 15:39:46.975241 25220 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:39:46.968229 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49930 (local address 127.3.89.193:36399)
0809 15:39:46.968656 (+ 427us) server_negotiation.cc:206] Beginning negotiation
0809 15:39:46.968669 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0809 15:39:46.968702 (+ 33us) server_negotiation.cc:407] Connection header received
0809 15:39:46.968986 (+ 284us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 15:39:46.968996 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 15:39:46.969210 (+ 214us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 15:39:46.969382 (+ 172us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 15:39:46.969912 (+ 530us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:39:46.971105 (+ 1193us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:39:46.972852 (+ 1747us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:39:46.973648 (+ 796us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:39:46.973753 (+ 105us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:39:46.973928 (+ 175us) server_negotiation.cc:1035] Waiting for connection context
0809 15:39:46.974774 (+ 846us) server_negotiation.cc:299] Negotiation successful
0809 15:39:46.974902 (+ 128us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":335,"thread_start_us":124,"threads_started":1}
I20260809 15:39:46.975620 25222 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:39:46.969022 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49030 (local address 127.3.89.195:41135)
0809 15:39:46.969870 (+ 848us) server_negotiation.cc:206] Beginning negotiation
0809 15:39:46.969881 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0809 15:39:46.969903 (+ 22us) server_negotiation.cc:407] Connection header received
0809 15:39:46.969986 (+ 83us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 15:39:46.969990 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 15:39:46.970173 (+ 183us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 15:39:46.970323 (+ 150us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 15:39:46.970613 (+ 290us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:39:46.971606 (+ 993us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:39:46.972900 (+ 1294us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:39:46.973648 (+ 748us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:39:46.973753 (+ 105us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:39:46.973930 (+ 177us) server_negotiation.cc:1035] Waiting for connection context
0809 15:39:46.975128 (+ 1198us) server_negotiation.cc:299] Negotiation successful
0809 15:39:46.975232 (+ 104us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":737,"thread_start_us":164,"threads_started":1}
I20260809 15:39:46.977084 24987 tablet_service.cc:1467] Processing CreateTablet for tablet 8236f0c57e28476f8d66693e3017fc6c (DEFAULT_TABLE table=test-table [id=ee1edd9511dd42a1b291789ba34c8b97]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 15:39:46.978000 24987 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 8236f0c57e28476f8d66693e3017fc6c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 15:39:46.980175 25132 tablet_service.cc:1467] Processing CreateTablet for tablet 8236f0c57e28476f8d66693e3017fc6c (DEFAULT_TABLE table=test-table [id=ee1edd9511dd42a1b291789ba34c8b97]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 15:39:46.980636 24840 tablet_service.cc:1467] Processing CreateTablet for tablet 8236f0c57e28476f8d66693e3017fc6c (DEFAULT_TABLE table=test-table [id=ee1edd9511dd42a1b291789ba34c8b97]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 15:39:46.981233 25132 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 8236f0c57e28476f8d66693e3017fc6c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 15:39:46.981600 24840 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 8236f0c57e28476f8d66693e3017fc6c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 15:39:46.984563 25224 tablet_bootstrap.cc:492] T 8236f0c57e28476f8d66693e3017fc6c P 01481569ef2c4a0c9c5e72569fd73b95: Bootstrap starting.
I20260809 15:39:46.986577 25224 tablet_bootstrap.cc:654] T 8236f0c57e28476f8d66693e3017fc6c P 01481569ef2c4a0c9c5e72569fd73b95: Neither blocks nor log segments found. Creating new log.
I20260809 15:39:46.987367 25224 log.cc:824] T 8236f0c57e28476f8d66693e3017fc6c P 01481569ef2c4a0c9c5e72569fd73b95: Log is configured to *not* fsync() on all Append() calls
I20260809 15:39:46.988265 25226 tablet_bootstrap.cc:492] T 8236f0c57e28476f8d66693e3017fc6c P 5109319448fd46689c9f6b6c07ad9af4: Bootstrap starting.
I20260809 15:39:46.989239 25227 tablet_bootstrap.cc:492] T 8236f0c57e28476f8d66693e3017fc6c P 2f5c45c1f3c449d7a50861b9f201bb25: Bootstrap starting.
I20260809 15:39:46.989517 25224 tablet_bootstrap.cc:492] T 8236f0c57e28476f8d66693e3017fc6c P 01481569ef2c4a0c9c5e72569fd73b95: No bootstrap required, opened a new log
I20260809 15:39:46.989711 25224 ts_tablet_manager.cc:1397] T 8236f0c57e28476f8d66693e3017fc6c P 01481569ef2c4a0c9c5e72569fd73b95: Time spent bootstrapping tablet: real 0.005s user 0.004s sys 0.000s
I20260809 15:39:46.990914 25226 tablet_bootstrap.cc:654] T 8236f0c57e28476f8d66693e3017fc6c P 5109319448fd46689c9f6b6c07ad9af4: Neither blocks nor log segments found. Creating new log.
I20260809 15:39:46.991106 25227 tablet_bootstrap.cc:654] T 8236f0c57e28476f8d66693e3017fc6c P 2f5c45c1f3c449d7a50861b9f201bb25: Neither blocks nor log segments found. Creating new log.
I20260809 15:39:46.991770 25227 log.cc:824] T 8236f0c57e28476f8d66693e3017fc6c P 2f5c45c1f3c449d7a50861b9f201bb25: Log is configured to *not* fsync() on all Append() calls
I20260809 15:39:46.991782 25226 log.cc:824] T 8236f0c57e28476f8d66693e3017fc6c P 5109319448fd46689c9f6b6c07ad9af4: Log is configured to *not* fsync() on all Append() calls
I20260809 15:39:46.993649 25227 tablet_bootstrap.cc:492] T 8236f0c57e28476f8d66693e3017fc6c P 2f5c45c1f3c449d7a50861b9f201bb25: No bootstrap required, opened a new log
I20260809 15:39:46.993650 25226 tablet_bootstrap.cc:492] T 8236f0c57e28476f8d66693e3017fc6c P 5109319448fd46689c9f6b6c07ad9af4: No bootstrap required, opened a new log
I20260809 15:39:46.993639 25224 raft_consensus.cc:348] T 8236f0c57e28476f8d66693e3017fc6c P 01481569ef2c4a0c9c5e72569fd73b95 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5109319448fd46689c9f6b6c07ad9af4" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 36399 } } peers { permanent_uuid: "2f5c45c1f3c449d7a50861b9f201bb25" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41135 } } peers { permanent_uuid: "01481569ef2c4a0c9c5e72569fd73b95" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 38127 } }
I20260809 15:39:46.993839 25227 ts_tablet_manager.cc:1397] T 8236f0c57e28476f8d66693e3017fc6c P 2f5c45c1f3c449d7a50861b9f201bb25: Time spent bootstrapping tablet: real 0.005s user 0.000s sys 0.003s
I20260809 15:39:46.993839 25226 ts_tablet_manager.cc:1397] T 8236f0c57e28476f8d66693e3017fc6c P 5109319448fd46689c9f6b6c07ad9af4: Time spent bootstrapping tablet: real 0.006s user 0.001s sys 0.002s
I20260809 15:39:46.993925 25224 raft_consensus.cc:374] T 8236f0c57e28476f8d66693e3017fc6c P 01481569ef2c4a0c9c5e72569fd73b95 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 15:39:46.993991 25224 raft_consensus.cc:729] T 8236f0c57e28476f8d66693e3017fc6c P 01481569ef2c4a0c9c5e72569fd73b95 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 01481569ef2c4a0c9c5e72569fd73b95, State: Initialized, Role: FOLLOWER
I20260809 15:39:46.994419 25224 consensus_queue.cc:260] T 8236f0c57e28476f8d66693e3017fc6c P 01481569ef2c4a0c9c5e72569fd73b95 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5109319448fd46689c9f6b6c07ad9af4" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 36399 } } peers { permanent_uuid: "2f5c45c1f3c449d7a50861b9f201bb25" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41135 } } peers { permanent_uuid: "01481569ef2c4a0c9c5e72569fd73b95" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 38127 } }
I20260809 15:39:46.995072 25060 heartbeater.cc:503] Master 127.3.89.254:36375 was elected leader, sending a full tablet report...
I20260809 15:39:46.995421 25224 ts_tablet_manager.cc:1428] T 8236f0c57e28476f8d66693e3017fc6c P 01481569ef2c4a0c9c5e72569fd73b95: Time spent starting tablet: real 0.006s user 0.006s sys 0.000s
W20260809 15:39:46.996946 25061 tablet.cc:2369] T 8236f0c57e28476f8d66693e3017fc6c P 01481569ef2c4a0c9c5e72569fd73b95: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260809 15:39:46.998011 25227 raft_consensus.cc:348] T 8236f0c57e28476f8d66693e3017fc6c P 2f5c45c1f3c449d7a50861b9f201bb25 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5109319448fd46689c9f6b6c07ad9af4" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 36399 } } peers { permanent_uuid: "2f5c45c1f3c449d7a50861b9f201bb25" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41135 } } peers { permanent_uuid: "01481569ef2c4a0c9c5e72569fd73b95" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 38127 } }
I20260809 15:39:46.998095 25226 raft_consensus.cc:348] T 8236f0c57e28476f8d66693e3017fc6c P 5109319448fd46689c9f6b6c07ad9af4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5109319448fd46689c9f6b6c07ad9af4" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 36399 } } peers { permanent_uuid: "2f5c45c1f3c449d7a50861b9f201bb25" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41135 } } peers { permanent_uuid: "01481569ef2c4a0c9c5e72569fd73b95" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 38127 } }
I20260809 15:39:46.998282 25226 raft_consensus.cc:374] T 8236f0c57e28476f8d66693e3017fc6c P 5109319448fd46689c9f6b6c07ad9af4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 15:39:46.998265 25227 raft_consensus.cc:374] T 8236f0c57e28476f8d66693e3017fc6c P 2f5c45c1f3c449d7a50861b9f201bb25 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 15:39:46.998334 25226 raft_consensus.cc:729] T 8236f0c57e28476f8d66693e3017fc6c P 5109319448fd46689c9f6b6c07ad9af4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5109319448fd46689c9f6b6c07ad9af4, State: Initialized, Role: FOLLOWER
I20260809 15:39:46.998334 25227 raft_consensus.cc:729] T 8236f0c57e28476f8d66693e3017fc6c P 2f5c45c1f3c449d7a50861b9f201bb25 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2f5c45c1f3c449d7a50861b9f201bb25, State: Initialized, Role: FOLLOWER
I20260809 15:39:46.998746 25227 consensus_queue.cc:260] T 8236f0c57e28476f8d66693e3017fc6c P 2f5c45c1f3c449d7a50861b9f201bb25 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5109319448fd46689c9f6b6c07ad9af4" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 36399 } } peers { permanent_uuid: "2f5c45c1f3c449d7a50861b9f201bb25" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41135 } } peers { permanent_uuid: "01481569ef2c4a0c9c5e72569fd73b95" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 38127 } }
I20260809 15:39:46.998746 25226 consensus_queue.cc:260] T 8236f0c57e28476f8d66693e3017fc6c P 5109319448fd46689c9f6b6c07ad9af4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5109319448fd46689c9f6b6c07ad9af4" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 36399 } } peers { permanent_uuid: "2f5c45c1f3c449d7a50861b9f201bb25" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41135 } } peers { permanent_uuid: "01481569ef2c4a0c9c5e72569fd73b95" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 38127 } }
I20260809 15:39:46.999612 24914 heartbeater.cc:503] Master 127.3.89.254:36375 was elected leader, sending a full tablet report...
I20260809 15:39:46.999661 25226 ts_tablet_manager.cc:1428] T 8236f0c57e28476f8d66693e3017fc6c P 5109319448fd46689c9f6b6c07ad9af4: Time spent starting tablet: real 0.006s user 0.002s sys 0.004s
I20260809 15:39:46.999661 25227 ts_tablet_manager.cc:1428] T 8236f0c57e28476f8d66693e3017fc6c P 2f5c45c1f3c449d7a50861b9f201bb25: Time spent starting tablet: real 0.006s user 0.003s sys 0.003s
I20260809 15:39:46.999948 25207 heartbeater.cc:503] Master 127.3.89.254:36375 was elected leader, sending a full tablet report...
W20260809 15:39:47.044894 24915 tablet.cc:2369] T 8236f0c57e28476f8d66693e3017fc6c P 5109319448fd46689c9f6b6c07ad9af4: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260809 15:39:47.089401 25231 raft_consensus.cc:482] T 8236f0c57e28476f8d66693e3017fc6c P 5109319448fd46689c9f6b6c07ad9af4 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 15:39:47.089605 25231 raft_consensus.cc:504] T 8236f0c57e28476f8d66693e3017fc6c P 5109319448fd46689c9f6b6c07ad9af4 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5109319448fd46689c9f6b6c07ad9af4" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 36399 } } peers { permanent_uuid: "2f5c45c1f3c449d7a50861b9f201bb25" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41135 } } peers { permanent_uuid: "01481569ef2c4a0c9c5e72569fd73b95" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 38127 } }
I20260809 15:39:47.090742 25231 leader_election.cc:288] T 8236f0c57e28476f8d66693e3017fc6c P 5109319448fd46689c9f6b6c07ad9af4 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 2f5c45c1f3c449d7a50861b9f201bb25 (127.3.89.195:41135), 01481569ef2c4a0c9c5e72569fd73b95 (127.3.89.194:38127)
I20260809 15:39:47.094486 25222 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:39:47.091095 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.89.193:40987 (local address 127.3.89.195:41135)
0809 15:39:47.091232 (+ 137us) server_negotiation.cc:206] Beginning negotiation
0809 15:39:47.091237 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 15:39:47.091340 (+ 103us) server_negotiation.cc:407] Connection header received
0809 15:39:47.091525 (+ 185us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 15:39:47.091527 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 15:39:47.091568 (+ 41us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 15:39:47.091656 (+ 88us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 15:39:47.091944 (+ 288us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:39:47.092444 (+ 500us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:39:47.093521 (+ 1077us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:39:47.093933 (+ 412us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:39:47.094061 (+ 128us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:39:47.094112 (+ 51us) server_negotiation.cc:1035] Waiting for connection context
0809 15:39:47.094301 (+ 189us) server_negotiation.cc:299] Negotiation successful
0809 15:39:47.094368 (+ 67us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":55}
I20260809 15:39:47.094511 25233 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:39:47.090919 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.89.195:41135 (local address 127.3.89.193:40987)
0809 15:39:47.091230 (+ 311us) negotiation.cc:107] Waiting for socket to connect
0809 15:39:47.091247 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0809 15:39:47.091384 (+ 137us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 15:39:47.091668 (+ 284us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 15:39:47.091671 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 15:39:47.091715 (+ 44us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 15:39:47.091830 (+ 115us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:39:47.091835 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:39:47.092555 (+ 720us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:39:47.092561 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:39:47.093394 (+ 833us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:39:47.093404 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:39:47.094061 (+ 657us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:39:47.094065 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:39:47.094160 (+ 95us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:39:47.094188 (+ 28us) client_negotiation.cc:769] Sending connection context
0809 15:39:47.094270 (+ 82us) client_negotiation.cc:240] Negotiation successful
0809 15:39:47.094350 (+ 80us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":220,"mutex_wait_us":3,"spinlock_wait_cycles":7296,"thread_start_us":126,"threads_started":1}
I20260809 15:39:47.095005 25223 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:39:47.091305 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.3.89.193:40857 (local address 127.3.89.194:38127)
0809 15:39:47.091426 (+ 121us) server_negotiation.cc:206] Beginning negotiation
0809 15:39:47.091429 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0809 15:39:47.091663 (+ 234us) server_negotiation.cc:407] Connection header received
0809 15:39:47.091849 (+ 186us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 15:39:47.091852 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 15:39:47.091896 (+ 44us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 15:39:47.091982 (+ 86us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 15:39:47.092267 (+ 285us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:39:47.092751 (+ 484us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:39:47.093835 (+ 1084us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:39:47.094221 (+ 386us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:39:47.094351 (+ 130us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:39:47.094396 (+ 45us) server_negotiation.cc:1035] Waiting for connection context
0809 15:39:47.094762 (+ 366us) server_negotiation.cc:299] Negotiation successful
0809 15:39:47.094844 (+ 82us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":33}
I20260809 15:39:47.095178 25154 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8236f0c57e28476f8d66693e3017fc6c" candidate_uuid: "5109319448fd46689c9f6b6c07ad9af4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2f5c45c1f3c449d7a50861b9f201bb25" is_pre_election: true
I20260809 15:39:47.095430 25234 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:39:47.091090 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.89.194:38127 (local address 127.3.89.193:40857)
0809 15:39:47.091557 (+ 467us) negotiation.cc:107] Waiting for socket to connect
0809 15:39:47.091569 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0809 15:39:47.091726 (+ 157us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 15:39:47.091997 (+ 271us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 15:39:47.091999 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 15:39:47.092016 (+ 17us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 15:39:47.092147 (+ 131us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:39:47.092152 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:39:47.092872 (+ 720us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:39:47.092878 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:39:47.093702 (+ 824us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:39:47.093711 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:39:47.094352 (+ 641us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:39:47.094355 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:39:47.094461 (+ 106us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:39:47.094479 (+ 18us) client_negotiation.cc:769] Sending connection context
0809 15:39:47.095159 (+ 680us) client_negotiation.cc:240] Negotiation successful
0809 15:39:47.095237 (+ 78us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":411,"mutex_wait_us":37,"spinlock_wait_cycles":26240,"thread_start_us":88,"threads_started":1}
I20260809 15:39:47.095498 25154 raft_consensus.cc:2454] T 8236f0c57e28476f8d66693e3017fc6c P 2f5c45c1f3c449d7a50861b9f201bb25 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5109319448fd46689c9f6b6c07ad9af4 in term 0.
I20260809 15:39:47.095999 24791 leader_election.cc:302] T 8236f0c57e28476f8d66693e3017fc6c P 5109319448fd46689c9f6b6c07ad9af4 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2f5c45c1f3c449d7a50861b9f201bb25, 5109319448fd46689c9f6b6c07ad9af4; no voters:
I20260809 15:39:47.096104 25010 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8236f0c57e28476f8d66693e3017fc6c" candidate_uuid: "5109319448fd46689c9f6b6c07ad9af4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "01481569ef2c4a0c9c5e72569fd73b95" is_pre_election: true
I20260809 15:39:47.096292 25231 raft_consensus.cc:2781] T 8236f0c57e28476f8d66693e3017fc6c P 5109319448fd46689c9f6b6c07ad9af4 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260809 15:39:47.096382 25231 raft_consensus.cc:482] T 8236f0c57e28476f8d66693e3017fc6c P 5109319448fd46689c9f6b6c07ad9af4 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 15:39:47.096436 25231 raft_consensus.cc:3037] T 8236f0c57e28476f8d66693e3017fc6c P 5109319448fd46689c9f6b6c07ad9af4 [term 0 FOLLOWER]: Advancing to term 1
I20260809 15:39:47.096444 25010 raft_consensus.cc:2454] T 8236f0c57e28476f8d66693e3017fc6c P 01481569ef2c4a0c9c5e72569fd73b95 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5109319448fd46689c9f6b6c07ad9af4 in term 0.
I20260809 15:39:47.097515 25231 raft_consensus.cc:504] T 8236f0c57e28476f8d66693e3017fc6c P 5109319448fd46689c9f6b6c07ad9af4 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5109319448fd46689c9f6b6c07ad9af4" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 36399 } } peers { permanent_uuid: "2f5c45c1f3c449d7a50861b9f201bb25" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41135 } } peers { permanent_uuid: "01481569ef2c4a0c9c5e72569fd73b95" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 38127 } }
I20260809 15:39:47.097832 25231 leader_election.cc:288] T 8236f0c57e28476f8d66693e3017fc6c P 5109319448fd46689c9f6b6c07ad9af4 [CANDIDATE]: Term 1 election: Requested vote from peers 2f5c45c1f3c449d7a50861b9f201bb25 (127.3.89.195:41135), 01481569ef2c4a0c9c5e72569fd73b95 (127.3.89.194:38127)
I20260809 15:39:47.098142 25154 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8236f0c57e28476f8d66693e3017fc6c" candidate_uuid: "5109319448fd46689c9f6b6c07ad9af4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2f5c45c1f3c449d7a50861b9f201bb25"
I20260809 15:39:47.098173 25010 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8236f0c57e28476f8d66693e3017fc6c" candidate_uuid: "5109319448fd46689c9f6b6c07ad9af4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "01481569ef2c4a0c9c5e72569fd73b95"
I20260809 15:39:47.098270 25154 raft_consensus.cc:3037] T 8236f0c57e28476f8d66693e3017fc6c P 2f5c45c1f3c449d7a50861b9f201bb25 [term 0 FOLLOWER]: Advancing to term 1
I20260809 15:39:47.098270 25010 raft_consensus.cc:3037] T 8236f0c57e28476f8d66693e3017fc6c P 01481569ef2c4a0c9c5e72569fd73b95 [term 0 FOLLOWER]: Advancing to term 1
I20260809 15:39:47.099538 25010 raft_consensus.cc:2454] T 8236f0c57e28476f8d66693e3017fc6c P 01481569ef2c4a0c9c5e72569fd73b95 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5109319448fd46689c9f6b6c07ad9af4 in term 1.
I20260809 15:39:47.099538 25154 raft_consensus.cc:2454] T 8236f0c57e28476f8d66693e3017fc6c P 2f5c45c1f3c449d7a50861b9f201bb25 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5109319448fd46689c9f6b6c07ad9af4 in term 1.
I20260809 15:39:47.099917 24788 leader_election.cc:302] T 8236f0c57e28476f8d66693e3017fc6c P 5109319448fd46689c9f6b6c07ad9af4 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 01481569ef2c4a0c9c5e72569fd73b95, 5109319448fd46689c9f6b6c07ad9af4; no voters:
I20260809 15:39:47.100225 25231 raft_consensus.cc:2781] T 8236f0c57e28476f8d66693e3017fc6c P 5109319448fd46689c9f6b6c07ad9af4 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 15:39:47.100457 25231 raft_consensus.cc:686] T 8236f0c57e28476f8d66693e3017fc6c P 5109319448fd46689c9f6b6c07ad9af4 [term 1 LEADER]: Becoming Leader. State: Replica: 5109319448fd46689c9f6b6c07ad9af4, State: Running, Role: LEADER
I20260809 15:39:47.100776 25231 consensus_queue.cc:237] T 8236f0c57e28476f8d66693e3017fc6c P 5109319448fd46689c9f6b6c07ad9af4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5109319448fd46689c9f6b6c07ad9af4" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 36399 } } peers { permanent_uuid: "2f5c45c1f3c449d7a50861b9f201bb25" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41135 } } peers { permanent_uuid: "01481569ef2c4a0c9c5e72569fd73b95" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 38127 } }
I20260809 15:39:47.103848 24703 catalog_manager.cc:5495] T 8236f0c57e28476f8d66693e3017fc6c P 5109319448fd46689c9f6b6c07ad9af4 reported cstate change: term changed from 0 to 1, leader changed from <none> to 5109319448fd46689c9f6b6c07ad9af4 (127.3.89.193). New cstate: current_term: 1 leader_uuid: "5109319448fd46689c9f6b6c07ad9af4" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5109319448fd46689c9f6b6c07ad9af4" member_type: VOTER last_known_addr { host: "127.3.89.193" port: 36399 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "2f5c45c1f3c449d7a50861b9f201bb25" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41135 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "01481569ef2c4a0c9c5e72569fd73b95" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 38127 } health_report { overall_health: UNKNOWN } } }
I20260809 15:39:47.123837 25220 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:39:47.120170 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49932 (local address 127.3.89.193:36399)
0809 15:39:47.120287 (+ 117us) server_negotiation.cc:206] Beginning negotiation
0809 15:39:47.120292 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 15:39:47.120359 (+ 67us) server_negotiation.cc:407] Connection header received
0809 15:39:47.120570 (+ 211us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 15:39:47.120573 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 15:39:47.120634 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 15:39:47.120743 (+ 109us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 15:39:47.120999 (+ 256us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:39:47.121508 (+ 509us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:39:47.122387 (+ 879us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:39:47.122612 (+ 225us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:39:47.122719 (+ 107us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:39:47.123071 (+ 352us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 15:39:47.123442 (+ 371us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 15:39:47.123578 (+ 136us) server_negotiation.cc:1035] Waiting for connection context
0809 15:39:47.123643 (+ 65us) server_negotiation.cc:299] Negotiation successful
0809 15:39:47.123693 (+ 50us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":49}
I20260809 15:39:47.129215 25154 raft_consensus.cc:1261] T 8236f0c57e28476f8d66693e3017fc6c P 2f5c45c1f3c449d7a50861b9f201bb25 [term 1 FOLLOWER]: Refusing update from remote peer 5109319448fd46689c9f6b6c07ad9af4: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 15:39:47.129215 25010 raft_consensus.cc:1261] T 8236f0c57e28476f8d66693e3017fc6c P 01481569ef2c4a0c9c5e72569fd73b95 [term 1 FOLLOWER]: Refusing update from remote peer 5109319448fd46689c9f6b6c07ad9af4: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 15:39:47.129886 25231 consensus_queue.cc:1035] T 8236f0c57e28476f8d66693e3017fc6c P 5109319448fd46689c9f6b6c07ad9af4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2f5c45c1f3c449d7a50861b9f201bb25" member_type: VOTER last_known_addr { host: "127.3.89.195" port: 41135 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 15:39:47.130108 25235 consensus_queue.cc:1035] T 8236f0c57e28476f8d66693e3017fc6c P 5109319448fd46689c9f6b6c07ad9af4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "01481569ef2c4a0c9c5e72569fd73b95" member_type: VOTER last_known_addr { host: "127.3.89.194" port: 38127 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 15:39:47.137580 25239 mvcc.cc:205] Tried to move back new op lower bound from 7316643787274706944 to 7316643787167469568. Current Snapshot: MvccSnapshot[applied={T|T < 7316643787274706944}]
I20260809 15:39:47.139839 25242 mvcc.cc:205] Tried to move back new op lower bound from 7316643787274706944 to 7316643787167469568. Current Snapshot: MvccSnapshot[applied={T|T < 7316643787274706944}]
I20260809 15:39:47.146994 24703 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:34364:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 15:39:47.147174 24703 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:34364:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 15:39:47.149461 24703 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 499b41fa1e164a888219f48bead27d72: Sending DeleteTablet for 3 replicas of tablet 8236f0c57e28476f8d66693e3017fc6c
I20260809 15:39:47.150270 25132 tablet_service.cc:1514] Processing DeleteTablet for tablet 8236f0c57e28476f8d66693e3017fc6c with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 15:39:47 UTC) from {username='slave'} at 127.0.0.1:49030
I20260809 15:39:47.150339 24987 tablet_service.cc:1514] Processing DeleteTablet for tablet 8236f0c57e28476f8d66693e3017fc6c with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 15:39:47 UTC) from {username='slave'} at 127.0.0.1:50414
I20260809 15:39:47.150436 24840 tablet_service.cc:1514] Processing DeleteTablet for tablet 8236f0c57e28476f8d66693e3017fc6c with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 15:39:47 UTC) from {username='slave'} at 127.0.0.1:49930
I20260809 15:39:47.150880 25252 tablet_replica.cc:330] T 8236f0c57e28476f8d66693e3017fc6c P 5109319448fd46689c9f6b6c07ad9af4: stopping tablet replica
I20260809 15:39:47.151194 25252 raft_consensus.cc:2229] T 8236f0c57e28476f8d66693e3017fc6c P 5109319448fd46689c9f6b6c07ad9af4 [term 1 LEADER]: Raft consensus shutting down.
I20260809 15:39:47.151443 3431 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskm314YR/build/debug/bin/kudu with pid 24772
I20260809 15:39:47.153386 25250 tablet_replica.cc:330] T 8236f0c57e28476f8d66693e3017fc6c P 2f5c45c1f3c449d7a50861b9f201bb25: stopping tablet replica
I20260809 15:39:47.153638 25250 raft_consensus.cc:2229] T 8236f0c57e28476f8d66693e3017fc6c P 2f5c45c1f3c449d7a50861b9f201bb25 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 15:39:47.153901 25250 raft_consensus.cc:2258] T 8236f0c57e28476f8d66693e3017fc6c P 2f5c45c1f3c449d7a50861b9f201bb25 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 15:39:47.154157 25251 tablet_replica.cc:330] T 8236f0c57e28476f8d66693e3017fc6c P 01481569ef2c4a0c9c5e72569fd73b95: stopping tablet replica
I20260809 15:39:47.154403 25251 raft_consensus.cc:2229] T 8236f0c57e28476f8d66693e3017fc6c P 01481569ef2c4a0c9c5e72569fd73b95 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 15:39:47.154636 25251 raft_consensus.cc:2258] T 8236f0c57e28476f8d66693e3017fc6c P 01481569ef2c4a0c9c5e72569fd73b95 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 15:39:47.156080 25250 ts_tablet_manager.cc:1905] T 8236f0c57e28476f8d66693e3017fc6c P 2f5c45c1f3c449d7a50861b9f201bb25: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 15:39:47.156148 25251 ts_tablet_manager.cc:1905] T 8236f0c57e28476f8d66693e3017fc6c P 01481569ef2c4a0c9c5e72569fd73b95: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 15:39:47.159587 25251 ts_tablet_manager.cc:1918] T 8236f0c57e28476f8d66693e3017fc6c P 01481569ef2c4a0c9c5e72569fd73b95: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 15:39:47.159647 25250 ts_tablet_manager.cc:1918] T 8236f0c57e28476f8d66693e3017fc6c P 2f5c45c1f3c449d7a50861b9f201bb25: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 15:39:47.159704 25251 log.cc:1196] T 8236f0c57e28476f8d66693e3017fc6c P 01481569ef2c4a0c9c5e72569fd73b95: Deleting WAL directory at /tmp/dist-test-taskm314YR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786289732823074-3431-0/minicluster-data/ts-1/wal/wals/8236f0c57e28476f8d66693e3017fc6c
I20260809 15:39:47.159747 25250 log.cc:1196] T 8236f0c57e28476f8d66693e3017fc6c P 2f5c45c1f3c449d7a50861b9f201bb25: Deleting WAL directory at /tmp/dist-test-taskm314YR/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786289732823074-3431-0/minicluster-data/ts-2/wal/wals/8236f0c57e28476f8d66693e3017fc6c
I20260809 15:39:47.160081 25250 ts_tablet_manager.cc:1939] T 8236f0c57e28476f8d66693e3017fc6c P 2f5c45c1f3c449d7a50861b9f201bb25: Deleting consensus metadata
I20260809 15:39:47.160073 25251 ts_tablet_manager.cc:1939] T 8236f0c57e28476f8d66693e3017fc6c P 01481569ef2c4a0c9c5e72569fd73b95: Deleting consensus metadata
I20260809 15:39:47.161170 24685 catalog_manager.cc:4841] TS 01481569ef2c4a0c9c5e72569fd73b95 (127.3.89.194:38127): tablet 8236f0c57e28476f8d66693e3017fc6c (table test-table [id=ee1edd9511dd42a1b291789ba34c8b97]) successfully deleted
I20260809 15:39:47.161227 24689 catalog_manager.cc:4841] TS 2f5c45c1f3c449d7a50861b9f201bb25 (127.3.89.195:41135): tablet 8236f0c57e28476f8d66693e3017fc6c (table test-table [id=ee1edd9511dd42a1b291789ba34c8b97]) successfully deleted
W20260809 15:39:47.161556 24685 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.3.89.193:36399 (error 108)
I20260809 15:39:47.161830 3431 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskm314YR/build/debug/bin/kudu with pid 24917
I20260809 15:39:47.164646 25221 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:39:47.164284 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.89.193:36399 (local address 127.0.0.1:49940)
0809 15:39:47.164459 (+ 175us) negotiation.cc:107] Waiting for socket to connect
0809 15:39:47.164543 (+ 84us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.89.193:36399: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":58}
W20260809 15:39:47.164851 24685 catalog_manager.cc:4584] TS 5109319448fd46689c9f6b6c07ad9af4 (127.3.89.193:36399): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 8236f0c57e28476f8d66693e3017fc6c: Network error: Client connection negotiation failed: client connection to 127.3.89.193:36399: connect: Connection refused (error 111)
I20260809 15:39:47.168977 3431 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskm314YR/build/debug/bin/kudu with pid 25063
I20260809 15:39:47.175439 3431 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskm314YR/build/debug/bin/kudu with pid 24669
2026-08-09T15:39:47Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/5 (1088 ms)
[----------] 6 tests from EncryptionPolicyTest (10429 ms total)
[----------] 18 tests from AuthTokenIssuingTest
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/0
Loading random data
Initializing database '/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786289732823074-3431-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 15:39:47 dist-test-slave-2w3w krb5kdc[25256](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 15:39:47 dist-test-slave-2w3w krb5kdc[25256](info): set up 2 sockets
Aug 09 15:39:47 dist-test-slave-2w3w krb5kdc[25256](info): commencing operation
krb5kdc: starting...
W20260809 15:39:48.248705 3431 mini_kdc.cc:121] Time spent starting KDC: real 1.050s user 0.001s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786289732823074-3431-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786289732823074-3431-0/krb5kdc/test-user.keytab.
Aug 09 15:39:48 dist-test-slave-2w3w krb5kdc[25256](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786289988, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T15:39:48Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T15:39:48Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786289732823074-3431-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786289732823074-3431-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260809 15:39:48.373155 3431 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskm314YR/build/debug/bin/kudu
/tmp/dist-test-taskm314YR/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786289732823074-3431-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786289732823074-3431-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786289732823074-3431-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786289732823074-3431-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:39747
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786289732823074-3431-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:33325
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:39747
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786289732823074-3431-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786289732823074-3431-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786289732823074-3431-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 15:39:48.482658 25274 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 15:39:48.482904 25274 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 15:39:48.482944 25274 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 15:39:48.486856 25274 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 15:39:48.486913 25274 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 15:39:48.486932 25274 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 15:39:48.486948 25274 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 15:39:48.486980 25274 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 15:39:48.492213 25274 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:33325
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786289732823074-3431-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786289732823074-3431-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:39747
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786289732823074-3431-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:39747
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786289732823074-3431-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.25274
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786289732823074-3431-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 09 Aug 2026 14:46:55 UTC on ccycloud.quasar-yswkmf.root.comops.site
build id 4576540
I20260809 15:39:48.493139 25274 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 15:39:48.494060 25274 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 15:39:48.502314 25281 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 15:39:48.502449 25280 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 15:39:48.502517 25274 server_base.cc:1034] running on GCE node
W20260809 15:39:48.502471 25283 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 15:39:48.503260 25274 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 15:39:48.504240 25274 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 15:39:48.505400 25274 hybrid_clock.cc:648] HybridClock initialized: now 1786289988505370 us; error 43 us; skew 500 ppm
Aug 09 15:39:48 dist-test-slave-2w3w krb5kdc[25256](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786289988, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 15:39:48.508270 25274 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260809 15:39:48.509148 25274 webserver.cc:460] Webserver started at http://127.0.0.1:42065/ using document root <none> and password file <none>
I20260809 15:39:48.509694 25274 fs_manager.cc:359] Metadata directory not provided
I20260809 15:39:48.509758 25274 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 15:39:48.509974 25274 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 15:39:48.512065 25274 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskm314YR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786289732823074-3431-0/minicluster-data/master-0/data/instance:
uuid: "39f92e6a2e5a4f158ad49093704cb9bb"
format_stamp: "Formatted at 2026-08-09 15:39:48 on dist-test-slave-2w3w"
server_key: "a92619d01f46bdfa50c53a327f1cc1d8"
server_key_iv: "9120bf5b6f4419dbbb52a4e891a85f34"
server_key_version: "encryptionkey@0"
I20260809 15:39:48.512640 25274 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskm314YR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786289732823074-3431-0/minicluster-data/master-0/wal/instance:
uuid: "39f92e6a2e5a4f158ad49093704cb9bb"
format_stamp: "Formatted at 2026-08-09 15:39:48 on dist-test-slave-2w3w"
server_key: "a92619d01f46bdfa50c53a327f1cc1d8"
server_key_iv: "9120bf5b6f4419dbbb52a4e891a85f34"
server_key_version: "encryptionkey@0"
I20260809 15:39:48.516314 25274 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.004s sys 0.000s
I20260809 15:39:48.518586 25291 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 15:39:48.519788 25274 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260809 15:39:48.519886 25274 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskm314YR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786289732823074-3431-0/minicluster-data/master-0/data,/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786289732823074-3431-0/minicluster-data/master-0/wal
uuid: "39f92e6a2e5a4f158ad49093704cb9bb"
format_stamp: "Formatted at 2026-08-09 15:39:48 on dist-test-slave-2w3w"
server_key: "a92619d01f46bdfa50c53a327f1cc1d8"
server_key_iv: "9120bf5b6f4419dbbb52a4e891a85f34"
server_key_version: "encryptionkey@0"
I20260809 15:39:48.519979 25274 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskm314YR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786289732823074-3431-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskm314YR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786289732823074-3431-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskm314YR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786289732823074-3431-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 15:39:48.554132 25274 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 15:39:48.557368 25274 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 15:39:48.557518 25274 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 15:39:48.565989 25274 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:39747
I20260809 15:39:48.565994 25345 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:39747 every 8 connection(s)
I20260809 15:39:48.566900 25274 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskm314YR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786289732823074-3431-0/minicluster-data/master-0/data/info.pb
I20260809 15:39:48.569545 3431 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskm314YR/build/debug/bin/kudu as pid 25274
I20260809 15:39:48.569574 25346 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 15:39:48.569672 3431 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskm314YR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786289732823074-3431-0/minicluster-data/master-0/wal/instance
I20260809 15:39:48.569969 3431 external_mini_cluster.cc:1428] Setting key 830c33fa356c97d07aef10185536ebf2
I20260809 15:39:48.576828 25346 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 39f92e6a2e5a4f158ad49093704cb9bb: Bootstrap starting.
I20260809 15:39:48.579146 25346 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 39f92e6a2e5a4f158ad49093704cb9bb: Neither blocks nor log segments found. Creating new log.
Aug 09 15:39:48 dist-test-slave-2w3w krb5kdc[25256](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786289988, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.0.1@KRBTEST.COM
I20260809 15:39:48.580044 25346 log.cc:824] T 00000000000000000000000000000000 P 39f92e6a2e5a4f158ad49093704cb9bb: Log is configured to *not* fsync() on all Append() calls
I20260809 15:39:48.581784 25346 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 39f92e6a2e5a4f158ad49093704cb9bb: No bootstrap required, opened a new log
I20260809 15:39:48.584973 25349 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:39:48.571389 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42299 (local address 127.0.0.1:39747)
0809 15:39:48.571784 (+ 395us) server_negotiation.cc:206] Beginning negotiation
0809 15:39:48.571801 (+ 17us) server_negotiation.cc:399] Waiting for connection header
0809 15:39:48.571842 (+ 41us) server_negotiation.cc:407] Connection header received
0809 15:39:48.573070 (+ 1228us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 15:39:48.573095 (+ 25us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 15:39:48.573520 (+ 425us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 15:39:48.573876 (+ 356us) server_negotiation.cc:226] Negotiated authn=SASL
0809 15:39:48.574586 (+ 710us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:39:48.576136 (+ 1550us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:39:48.576998 (+ 862us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:39:48.577462 (+ 464us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:39:48.577636 (+ 174us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:39:48.580472 (+ 2836us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 15:39:48.580477 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 15:39:48.580488 (+ 11us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 15:39:48.580514 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0809 15:39:48.582201 (+ 1687us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 15:39:48.582700 (+ 499us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 15:39:48.582705 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 15:39:48.582709 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 15:39:48.582824 (+ 115us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 15:39:48.583230 (+ 406us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 15:39:48.583238 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 15:39:48.583239 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 15:39:48.583599 (+ 360us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 15:39:48.583832 (+ 233us) server_negotiation.cc:1035] Waiting for connection context
0809 15:39:48.584101 (+ 269us) server_negotiation.cc:299] Negotiation successful
0809 15:39:48.584308 (+ 207us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":255,"thread_start_us":110,"threads_started":1}
I20260809 15:39:48.585636 25346 raft_consensus.cc:348] T 00000000000000000000000000000000 P 39f92e6a2e5a4f158ad49093704cb9bb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "39f92e6a2e5a4f158ad49093704cb9bb" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39747 } }
I20260809 15:39:48.585911 25346 raft_consensus.cc:374] T 00000000000000000000000000000000 P 39f92e6a2e5a4f158ad49093704cb9bb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 15:39:48.585971 25346 raft_consensus.cc:729] T 00000000000000000000000000000000 P 39f92e6a2e5a4f158ad49093704cb9bb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 39f92e6a2e5a4f158ad49093704cb9bb, State: Initialized, Role: FOLLOWER
I20260809 15:39:48.586404 25346 consensus_queue.cc:260] T 00000000000000000000000000000000 P 39f92e6a2e5a4f158ad49093704cb9bb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "39f92e6a2e5a4f158ad49093704cb9bb" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39747 } }
I20260809 15:39:48.586553 25346 raft_consensus.cc:388] T 00000000000000000000000000000000 P 39f92e6a2e5a4f158ad49093704cb9bb [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 15:39:48.586616 25346 raft_consensus.cc:482] T 00000000000000000000000000000000 P 39f92e6a2e5a4f158ad49093704cb9bb [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 15:39:48.586706 25346 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 39f92e6a2e5a4f158ad49093704cb9bb [term 0 FOLLOWER]: Advancing to term 1
I20260809 15:39:48.587790 25346 raft_consensus.cc:504] T 00000000000000000000000000000000 P 39f92e6a2e5a4f158ad49093704cb9bb [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "39f92e6a2e5a4f158ad49093704cb9bb" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39747 } }
I20260809 15:39:48.588109 3431 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260809 15:39:48.588146 25346 leader_election.cc:302] T 00000000000000000000000000000000 P 39f92e6a2e5a4f158ad49093704cb9bb [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 39f92e6a2e5a4f158ad49093704cb9bb; no voters:
I20260809 15:39:48.588506 25346 leader_election.cc:288] T 00000000000000000000000000000000 P 39f92e6a2e5a4f158ad49093704cb9bb [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 15:39:48.588610 25353 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 39f92e6a2e5a4f158ad49093704cb9bb [term 1 FOLLOWER]: Leader election won for term 1
I20260809 15:39:48.588781 25353 raft_consensus.cc:686] T 00000000000000000000000000000000 P 39f92e6a2e5a4f158ad49093704cb9bb [term 1 LEADER]: Becoming Leader. State: Replica: 39f92e6a2e5a4f158ad49093704cb9bb, State: Running, Role: LEADER
I20260809 15:39:48.589059 25353 consensus_queue.cc:237] T 00000000000000000000000000000000 P 39f92e6a2e5a4f158ad49093704cb9bb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "39f92e6a2e5a4f158ad49093704cb9bb" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39747 } }
I20260809 15:39:48.589732 25346 sys_catalog.cc:564] T 00000000000000000000000000000000 P 39f92e6a2e5a4f158ad49093704cb9bb [sys.catalog]: configured and running, proceeding with master startup.
I20260809 15:39:48.590178 25353 sys_catalog.cc:455] T 00000000000000000000000000000000 P 39f92e6a2e5a4f158ad49093704cb9bb [sys.catalog]: SysCatalogTable state changed. Reason: New leader 39f92e6a2e5a4f158ad49093704cb9bb. Latest consensus state: current_term: 1 leader_uuid: "39f92e6a2e5a4f158ad49093704cb9bb" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "39f92e6a2e5a4f158ad49093704cb9bb" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39747 } } }
I20260809 15:39:48.590302 25353 sys_catalog.cc:458] T 00000000000000000000000000000000 P 39f92e6a2e5a4f158ad49093704cb9bb [sys.catalog]: This master's current role is: LEADER
I20260809 15:39:48.590691 25361 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 15:39:48.590665 25354 sys_catalog.cc:455] T 00000000000000000000000000000000 P 39f92e6a2e5a4f158ad49093704cb9bb [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "39f92e6a2e5a4f158ad49093704cb9bb" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "39f92e6a2e5a4f158ad49093704cb9bb" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39747 } } }
I20260809 15:39:48.590772 25354 sys_catalog.cc:458] T 00000000000000000000000000000000 P 39f92e6a2e5a4f158ad49093704cb9bb [sys.catalog]: This master's current role is: LEADER
I20260809 15:39:48.593812 25361 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 15:39:48.599535 25361 catalog_manager.cc:1347] Generated new cluster ID: a2028e59dd744a829864951082a68864
I20260809 15:39:48.599602 25361 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 15:39:48.604818 25349 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:39:48.595575 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44559 (local address 127.0.0.1:39747)
0809 15:39:48.595697 (+ 122us) server_negotiation.cc:206] Beginning negotiation
0809 15:39:48.595702 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 15:39:48.595964 (+ 262us) server_negotiation.cc:407] Connection header received
0809 15:39:48.596095 (+ 131us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 15:39:48.596098 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 15:39:48.596144 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 15:39:48.596252 (+ 108us) server_negotiation.cc:226] Negotiated authn=SASL
0809 15:39:48.596856 (+ 604us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:39:48.597580 (+ 724us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:39:48.598673 (+ 1093us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:39:48.599131 (+ 458us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:39:48.599239 (+ 108us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:39:48.601193 (+ 1954us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 15:39:48.601197 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 15:39:48.601200 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 15:39:48.601226 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0809 15:39:48.602888 (+ 1662us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 15:39:48.603466 (+ 578us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 15:39:48.603470 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 15:39:48.603472 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 15:39:48.603561 (+ 89us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 15:39:48.603906 (+ 345us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 15:39:48.603910 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 15:39:48.603911 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 15:39:48.604132 (+ 221us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 15:39:48.604298 (+ 166us) server_negotiation.cc:1035] Waiting for connection context
0809 15:39:48.604559 (+ 261us) server_negotiation.cc:299] Negotiation successful
0809 15:39:48.604662 (+ 103us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":42}
I20260809 15:39:48.612884 25361 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 15:39:48.614013 25361 catalog_manager.cc:1504] Loading token signing keys...
I20260809 15:39:48.621259 25361 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 39f92e6a2e5a4f158ad49093704cb9bb: Generated new TSK 0
I20260809 15:39:48.621996 25361 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260809 15:39:48.626690 3431 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskm314YR/build/debug/bin/kudu with pid 25274
W20260809 15:39:48.634997 25254 connection.cc:535] client connection to 127.0.0.1:39747 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:39747): Connection reset by peer (error 104)
2026-08-09T15:39:48Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/0 (1450 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/1
Loading random data
Initializing database '/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786289732823074-3431-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 15:39:48 dist-test-slave-2w3w krb5kdc[25384](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 15:39:48 dist-test-slave-2w3w krb5kdc[25384](info): set up 2 sockets
Aug 09 15:39:48 dist-test-slave-2w3w krb5kdc[25384](info): commencing operation
krb5kdc: starting...
W20260809 15:39:49.704526 3431 mini_kdc.cc:121] Time spent starting KDC: real 1.055s user 0.000s sys 0.008s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786289732823074-3431-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786289732823074-3431-0/krb5kdc/test-user.keytab.
Aug 09 15:39:49 dist-test-slave-2w3w krb5kdc[25384](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786289989, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T15:39:49Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T15:39:49Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786289732823074-3431-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786289732823074-3431-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260809 15:39:49.835781 3431 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskm314YR/build/debug/bin/kudu
/tmp/dist-test-taskm314YR/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786289732823074-3431-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786289732823074-3431-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786289732823074-3431-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786289732823074-3431-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:44469
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786289732823074-3431-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:44433
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:44469
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786289732823074-3431-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786289732823074-3431-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786289732823074-3431-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 15:39:49.947721 25398 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 15:39:49.948020 25398 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 15:39:49.948107 25398 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 15:39:49.952178 25398 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 15:39:49.952263 25398 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 15:39:49.952303 25398 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 15:39:49.952363 25398 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 15:39:49.952402 25398 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 15:39:49.957816 25398 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:44433
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786289732823074-3431-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786289732823074-3431-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:44469
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786289732823074-3431-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:44469
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786289732823074-3431-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.25398
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786289732823074-3431-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 09 Aug 2026 14:46:55 UTC on ccycloud.quasar-yswkmf.root.comops.site
build id 4576540
I20260809 15:39:49.958863 25398 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 15:39:49.959925 25398 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 15:39:49.968379 25407 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 15:39:49.968367 25405 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 15:39:49.968367 25403 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 15:39:49.968773 25398 server_base.cc:1034] running on GCE node
I20260809 15:39:49.969343 25398 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 15:39:49.970458 25398 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 15:39:49.971648 25398 hybrid_clock.cc:648] HybridClock initialized: now 1786289989971620 us; error 41 us; skew 500 ppm
Aug 09 15:39:49 dist-test-slave-2w3w krb5kdc[25384](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786289989, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 15:39:49.979544 25398 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260809 15:39:49.980484 25398 webserver.cc:460] Webserver started at http://127.0.0.1:35659/ using document root <none> and password file <none>
I20260809 15:39:49.981050 25398 fs_manager.cc:359] Metadata directory not provided
I20260809 15:39:49.981113 25398 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 15:39:49.981362 25398 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 15:39:49.983546 25398 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskm314YR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786289732823074-3431-0/minicluster-data/master-0/data/instance:
uuid: "d87f532f8fa64386b52e3361359fd728"
format_stamp: "Formatted at 2026-08-09 15:39:49 on dist-test-slave-2w3w"
server_key: "de236f1b7d2047db4db95ccee5075ba0"
server_key_iv: "5a3f506103bfd478faedb319e3783088"
server_key_version: "encryptionkey@0"
I20260809 15:39:49.984122 25398 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskm314YR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786289732823074-3431-0/minicluster-data/master-0/wal/instance:
uuid: "d87f532f8fa64386b52e3361359fd728"
format_stamp: "Formatted at 2026-08-09 15:39:49 on dist-test-slave-2w3w"
server_key: "de236f1b7d2047db4db95ccee5075ba0"
server_key_iv: "5a3f506103bfd478faedb319e3783088"
server_key_version: "encryptionkey@0"
I20260809 15:39:49.987926 25398 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.001s
I20260809 15:39:49.990367 25414 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 15:39:49.991624 25398 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.001s
I20260809 15:39:49.991762 25398 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskm314YR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786289732823074-3431-0/minicluster-data/master-0/data,/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786289732823074-3431-0/minicluster-data/master-0/wal
uuid: "d87f532f8fa64386b52e3361359fd728"
format_stamp: "Formatted at 2026-08-09 15:39:49 on dist-test-slave-2w3w"
server_key: "de236f1b7d2047db4db95ccee5075ba0"
server_key_iv: "5a3f506103bfd478faedb319e3783088"
server_key_version: "encryptionkey@0"
I20260809 15:39:49.991889 25398 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskm314YR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786289732823074-3431-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskm314YR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786289732823074-3431-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskm314YR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786289732823074-3431-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 15:39:50.004498 25398 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 15:39:50.007508 25398 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 15:39:50.007718 25398 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 15:39:50.014901 25398 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:44469
I20260809 15:39:50.015043 25473 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:44469 every 8 connection(s)
I20260809 15:39:50.015921 25398 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskm314YR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786289732823074-3431-0/minicluster-data/master-0/data/info.pb
I20260809 15:39:50.018628 25474 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 15:39:50.022737 3431 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskm314YR/build/debug/bin/kudu as pid 25398
I20260809 15:39:50.022857 3431 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskm314YR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786289732823074-3431-0/minicluster-data/master-0/wal/instance
I20260809 15:39:50.023150 3431 external_mini_cluster.cc:1428] Setting key f4094531570a6df1679376e4cf2d718a
I20260809 15:39:50.024516 25474 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d87f532f8fa64386b52e3361359fd728: Bootstrap starting.
I20260809 15:39:50.026935 25474 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P d87f532f8fa64386b52e3361359fd728: Neither blocks nor log segments found. Creating new log.
I20260809 15:39:50.027719 25474 log.cc:824] T 00000000000000000000000000000000 P d87f532f8fa64386b52e3361359fd728: Log is configured to *not* fsync() on all Append() calls
I20260809 15:39:50.029934 25474 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d87f532f8fa64386b52e3361359fd728: No bootstrap required, opened a new log
I20260809 15:39:50.032374 25477 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:39:50.024466 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46597 (local address 127.0.0.1:44469)
0809 15:39:50.024890 (+ 424us) server_negotiation.cc:206] Beginning negotiation
0809 15:39:50.024907 (+ 17us) server_negotiation.cc:399] Waiting for connection header
0809 15:39:50.024943 (+ 36us) server_negotiation.cc:407] Connection header received
0809 15:39:50.025572 (+ 629us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 15:39:50.025588 (+ 16us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 15:39:50.025915 (+ 327us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 15:39:50.026261 (+ 346us) server_negotiation.cc:226] Negotiated authn=SASL
0809 15:39:50.026588 (+ 327us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:39:50.027679 (+ 1091us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:39:50.028862 (+ 1183us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:39:50.029320 (+ 458us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:39:50.029451 (+ 131us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:39:50.030427 (+ 976us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 15:39:50.030432 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 15:39:50.030444 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 15:39:50.030854 (+ 410us) server_negotiation.cc:922] calling sasl_server_start()
0809 15:39:50.030910 (+ 56us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 15:39:50.030932 (+ 22us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 15:39:50.031098 (+ 166us) server_negotiation.cc:1035] Waiting for connection context
0809 15:39:50.031279 (+ 181us) server_negotiation.cc:299] Negotiation successful
0809 15:39:50.031433 (+ 154us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":280,"thread_start_us":140,"threads_started":1}
I20260809 15:39:50.033244 25474 raft_consensus.cc:348] T 00000000000000000000000000000000 P d87f532f8fa64386b52e3361359fd728 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d87f532f8fa64386b52e3361359fd728" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44469 } }
I20260809 15:39:50.033454 25474 raft_consensus.cc:374] T 00000000000000000000000000000000 P d87f532f8fa64386b52e3361359fd728 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 15:39:50.033540 25474 raft_consensus.cc:729] T 00000000000000000000000000000000 P d87f532f8fa64386b52e3361359fd728 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d87f532f8fa64386b52e3361359fd728, State: Initialized, Role: FOLLOWER
I20260809 15:39:50.033984 25474 consensus_queue.cc:260] T 00000000000000000000000000000000 P d87f532f8fa64386b52e3361359fd728 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d87f532f8fa64386b52e3361359fd728" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44469 } }
I20260809 15:39:50.034142 25474 raft_consensus.cc:388] T 00000000000000000000000000000000 P d87f532f8fa64386b52e3361359fd728 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 15:39:50.034237 25474 raft_consensus.cc:482] T 00000000000000000000000000000000 P d87f532f8fa64386b52e3361359fd728 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 15:39:50.034340 25474 raft_consensus.cc:3037] T 00000000000000000000000000000000 P d87f532f8fa64386b52e3361359fd728 [term 0 FOLLOWER]: Advancing to term 1
I20260809 15:39:50.035439 3431 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260809 15:39:50.035398 25474 raft_consensus.cc:504] T 00000000000000000000000000000000 P d87f532f8fa64386b52e3361359fd728 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d87f532f8fa64386b52e3361359fd728" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44469 } }
I20260809 15:39:50.035748 25474 leader_election.cc:302] T 00000000000000000000000000000000 P d87f532f8fa64386b52e3361359fd728 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: d87f532f8fa64386b52e3361359fd728; no voters:
I20260809 15:39:50.036089 25474 leader_election.cc:288] T 00000000000000000000000000000000 P d87f532f8fa64386b52e3361359fd728 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 15:39:50.036194 25480 raft_consensus.cc:2781] T 00000000000000000000000000000000 P d87f532f8fa64386b52e3361359fd728 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 15:39:50.036469 25480 raft_consensus.cc:686] T 00000000000000000000000000000000 P d87f532f8fa64386b52e3361359fd728 [term 1 LEADER]: Becoming Leader. State: Replica: d87f532f8fa64386b52e3361359fd728, State: Running, Role: LEADER
I20260809 15:39:50.036777 25480 consensus_queue.cc:237] T 00000000000000000000000000000000 P d87f532f8fa64386b52e3361359fd728 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d87f532f8fa64386b52e3361359fd728" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44469 } }
I20260809 15:39:50.037410 25474 sys_catalog.cc:564] T 00000000000000000000000000000000 P d87f532f8fa64386b52e3361359fd728 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 15:39:50.037945 25481 sys_catalog.cc:455] T 00000000000000000000000000000000 P d87f532f8fa64386b52e3361359fd728 [sys.catalog]: SysCatalogTable state changed. Reason: New leader d87f532f8fa64386b52e3361359fd728. Latest consensus state: current_term: 1 leader_uuid: "d87f532f8fa64386b52e3361359fd728" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d87f532f8fa64386b52e3361359fd728" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44469 } } }
I20260809 15:39:50.038080 25481 sys_catalog.cc:458] T 00000000000000000000000000000000 P d87f532f8fa64386b52e3361359fd728 [sys.catalog]: This master's current role is: LEADER
I20260809 15:39:50.038414 25482 sys_catalog.cc:455] T 00000000000000000000000000000000 P d87f532f8fa64386b52e3361359fd728 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "d87f532f8fa64386b52e3361359fd728" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d87f532f8fa64386b52e3361359fd728" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44469 } } }
I20260809 15:39:50.038532 25482 sys_catalog.cc:458] T 00000000000000000000000000000000 P d87f532f8fa64386b52e3361359fd728 [sys.catalog]: This master's current role is: LEADER
I20260809 15:39:50.038825 25491 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 15:39:50.042078 25491 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 15:39:50.043668 25477 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:39:50.037811 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40435 (local address 127.0.0.1:44469)
0809 15:39:50.037915 (+ 104us) server_negotiation.cc:206] Beginning negotiation
0809 15:39:50.037920 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 15:39:50.038568 (+ 648us) server_negotiation.cc:407] Connection header received
0809 15:39:50.038772 (+ 204us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 15:39:50.038775 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 15:39:50.038836 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 15:39:50.038892 (+ 56us) server_negotiation.cc:226] Negotiated authn=SASL
0809 15:39:50.039356 (+ 464us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:39:50.040097 (+ 741us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:39:50.041260 (+ 1163us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:39:50.041576 (+ 316us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:39:50.041650 (+ 74us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:39:50.043138 (+ 1488us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 15:39:50.043142 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 15:39:50.043145 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 15:39:50.043159 (+ 14us) server_negotiation.cc:922] calling sasl_server_start()
0809 15:39:50.043186 (+ 27us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 15:39:50.043198 (+ 12us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 15:39:50.043255 (+ 57us) server_negotiation.cc:1035] Waiting for connection context
0809 15:39:50.043494 (+ 239us) server_negotiation.cc:299] Negotiation successful
0809 15:39:50.043534 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":29}
I20260809 15:39:50.048028 25491 catalog_manager.cc:1347] Generated new cluster ID: 622481ba866448448cd878152fefbfa6
I20260809 15:39:50.048094 25491 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 15:39:50.060148 25491 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 15:39:50.061314 25491 catalog_manager.cc:1504] Loading token signing keys...
I20260809 15:39:50.073580 25491 catalog_manager.cc:5868] T 00000000000000000000000000000000 P d87f532f8fa64386b52e3361359fd728: Generated new TSK 0
I20260809 15:39:50.074252 25491 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260809 15:39:50.086717 3431 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskm314YR/build/debug/bin/kudu with pid 25398
W20260809 15:39:50.095142 25381 connection.cc:535] client connection to 127.0.0.1:44469 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:44469): Connection reset by peer (error 104)
2026-08-09T15:39:50Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/1 (1460 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/2
Loading random data
Initializing database '/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786289732823074-3431-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 15:39:50 dist-test-slave-2w3w krb5kdc[25510](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 15:39:50 dist-test-slave-2w3w krb5kdc[25510](info): set up 2 sockets
Aug 09 15:39:50 dist-test-slave-2w3w krb5kdc[25510](info): commencing operation
krb5kdc: starting...
W20260809 15:39:51.174167 3431 mini_kdc.cc:121] Time spent starting KDC: real 1.064s user 0.001s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786289732823074-3431-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786289732823074-3431-0/krb5kdc/test-user.keytab.
Aug 09 15:39:51 dist-test-slave-2w3w krb5kdc[25510](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786289991, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T15:39:51Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T15:39:51Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786289732823074-3431-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786289732823074-3431-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260809 15:39:51.301836 3431 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskm314YR/build/debug/bin/kudu
/tmp/dist-test-taskm314YR/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786289732823074-3431-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786289732823074-3431-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786289732823074-3431-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786289732823074-3431-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:42233
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786289732823074-3431-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:43927
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:42233
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786289732823074-3431-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786289732823074-3431-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786289732823074-3431-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 15:39:51.412532 25528 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 15:39:51.412853 25528 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 15:39:51.412935 25528 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 15:39:51.416960 25528 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 15:39:51.417057 25528 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 15:39:51.417095 25528 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 15:39:51.417133 25528 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 15:39:51.417169 25528 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 15:39:51.422475 25528 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:43927
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786289732823074-3431-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786289732823074-3431-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:42233
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786289732823074-3431-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:42233
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786289732823074-3431-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.25528
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786289732823074-3431-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 09 Aug 2026 14:46:55 UTC on ccycloud.quasar-yswkmf.root.comops.site
build id 4576540
I20260809 15:39:51.423583 25528 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 15:39:51.424613 25528 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 15:39:51.433247 25537 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 15:39:51.433239 25535 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 15:39:51.433239 25534 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 15:39:51.433594 25528 server_base.cc:1034] running on GCE node
I20260809 15:39:51.434204 25528 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 15:39:51.435362 25528 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 15:39:51.436523 25528 hybrid_clock.cc:648] HybridClock initialized: now 1786289991436496 us; error 42 us; skew 500 ppm
Aug 09 15:39:51 dist-test-slave-2w3w krb5kdc[25510](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786289991, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 15:39:51.439379 25528 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260809 15:39:51.440327 25528 webserver.cc:460] Webserver started at http://127.0.0.1:41145/ using document root <none> and password file <none>
I20260809 15:39:51.440837 25528 fs_manager.cc:359] Metadata directory not provided
I20260809 15:39:51.440896 25528 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 15:39:51.441061 25528 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 15:39:51.443073 25528 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskm314YR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786289732823074-3431-0/minicluster-data/master-0/data/instance:
uuid: "1aa385c544c4407ab86efba130a08724"
format_stamp: "Formatted at 2026-08-09 15:39:51 on dist-test-slave-2w3w"
server_key: "d99ccf9cad56089c2e5bd8bbc2a6eb54"
server_key_iv: "dffb7e0249c39a0e1ac62c6e5e785fd6"
server_key_version: "encryptionkey@0"
I20260809 15:39:51.443604 25528 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskm314YR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786289732823074-3431-0/minicluster-data/master-0/wal/instance:
uuid: "1aa385c544c4407ab86efba130a08724"
format_stamp: "Formatted at 2026-08-09 15:39:51 on dist-test-slave-2w3w"
server_key: "d99ccf9cad56089c2e5bd8bbc2a6eb54"
server_key_iv: "dffb7e0249c39a0e1ac62c6e5e785fd6"
server_key_version: "encryptionkey@0"
I20260809 15:39:51.447301 25528 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.002s sys 0.002s
I20260809 15:39:51.449838 25544 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 15:39:51.451177 25528 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260809 15:39:51.451306 25528 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskm314YR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786289732823074-3431-0/minicluster-data/master-0/data,/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786289732823074-3431-0/minicluster-data/master-0/wal
uuid: "1aa385c544c4407ab86efba130a08724"
format_stamp: "Formatted at 2026-08-09 15:39:51 on dist-test-slave-2w3w"
server_key: "d99ccf9cad56089c2e5bd8bbc2a6eb54"
server_key_iv: "dffb7e0249c39a0e1ac62c6e5e785fd6"
server_key_version: "encryptionkey@0"
I20260809 15:39:51.451417 25528 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskm314YR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786289732823074-3431-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskm314YR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786289732823074-3431-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskm314YR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786289732823074-3431-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 15:39:51.456094 25528 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 15:39:51.459316 25528 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 15:39:51.459519 25528 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 15:39:51.466768 25528 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:42233
I20260809 15:39:51.466797 25602 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:42233 every 8 connection(s)
I20260809 15:39:51.467751 25528 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskm314YR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786289732823074-3431-0/minicluster-data/master-0/data/info.pb
I20260809 15:39:51.470443 25603 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 15:39:51.476058 25603 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1aa385c544c4407ab86efba130a08724: Bootstrap starting.
I20260809 15:39:51.478060 3431 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskm314YR/build/debug/bin/kudu as pid 25528
I20260809 15:39:51.478200 3431 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskm314YR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786289732823074-3431-0/minicluster-data/master-0/wal/instance
I20260809 15:39:51.478497 3431 external_mini_cluster.cc:1428] Setting key f3b6e5b6877c22b60471f291e88cc17e
I20260809 15:39:51.478789 25603 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 1aa385c544c4407ab86efba130a08724: Neither blocks nor log segments found. Creating new log.
I20260809 15:39:51.479697 25603 log.cc:824] T 00000000000000000000000000000000 P 1aa385c544c4407ab86efba130a08724: Log is configured to *not* fsync() on all Append() calls
I20260809 15:39:51.481593 25603 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1aa385c544c4407ab86efba130a08724: No bootstrap required, opened a new log
I20260809 15:39:51.483767 25606 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:39:51.479718 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42929 (local address 127.0.0.1:42233)
0809 15:39:51.480233 (+ 515us) server_negotiation.cc:206] Beginning negotiation
0809 15:39:51.480255 (+ 22us) server_negotiation.cc:399] Waiting for connection header
0809 15:39:51.480313 (+ 58us) server_negotiation.cc:407] Connection header received
0809 15:39:51.481020 (+ 707us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 15:39:51.481038 (+ 18us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 15:39:51.481307 (+ 269us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 15:39:51.481640 (+ 333us) server_negotiation.cc:226] Negotiated authn=SASL
0809 15:39:51.482436 (+ 796us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 15:39:51.482441 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 15:39:51.482461 (+ 20us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 15:39:51.482471 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0809 15:39:51.482525 (+ 54us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 15:39:51.482543 (+ 18us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 15:39:51.482659 (+ 116us) server_negotiation.cc:1035] Waiting for connection context
0809 15:39:51.482768 (+ 109us) server_negotiation.cc:299] Negotiation successful
0809 15:39:51.483100 (+ 332us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":343,"thread_start_us":151,"threads_started":1}
I20260809 15:39:51.485659 25603 raft_consensus.cc:348] T 00000000000000000000000000000000 P 1aa385c544c4407ab86efba130a08724 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1aa385c544c4407ab86efba130a08724" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42233 } }
I20260809 15:39:51.485932 25603 raft_consensus.cc:374] T 00000000000000000000000000000000 P 1aa385c544c4407ab86efba130a08724 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 15:39:51.486016 25603 raft_consensus.cc:729] T 00000000000000000000000000000000 P 1aa385c544c4407ab86efba130a08724 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1aa385c544c4407ab86efba130a08724, State: Initialized, Role: FOLLOWER
I20260809 15:39:51.486428 25603 consensus_queue.cc:260] T 00000000000000000000000000000000 P 1aa385c544c4407ab86efba130a08724 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1aa385c544c4407ab86efba130a08724" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42233 } }
I20260809 15:39:51.486591 25603 raft_consensus.cc:388] T 00000000000000000000000000000000 P 1aa385c544c4407ab86efba130a08724 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 15:39:51.486591 3431 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260809 15:39:51.486680 25603 raft_consensus.cc:482] T 00000000000000000000000000000000 P 1aa385c544c4407ab86efba130a08724 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 15:39:51.486779 25603 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 1aa385c544c4407ab86efba130a08724 [term 0 FOLLOWER]: Advancing to term 1
I20260809 15:39:51.487762 25603 raft_consensus.cc:504] T 00000000000000000000000000000000 P 1aa385c544c4407ab86efba130a08724 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1aa385c544c4407ab86efba130a08724" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42233 } }
I20260809 15:39:51.488128 25603 leader_election.cc:302] T 00000000000000000000000000000000 P 1aa385c544c4407ab86efba130a08724 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 1aa385c544c4407ab86efba130a08724; no voters:
I20260809 15:39:51.488490 25603 leader_election.cc:288] T 00000000000000000000000000000000 P 1aa385c544c4407ab86efba130a08724 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 15:39:51.488546 25613 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 1aa385c544c4407ab86efba130a08724 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 15:39:51.488740 25613 raft_consensus.cc:686] T 00000000000000000000000000000000 P 1aa385c544c4407ab86efba130a08724 [term 1 LEADER]: Becoming Leader. State: Replica: 1aa385c544c4407ab86efba130a08724, State: Running, Role: LEADER
I20260809 15:39:51.489040 25613 consensus_queue.cc:237] T 00000000000000000000000000000000 P 1aa385c544c4407ab86efba130a08724 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1aa385c544c4407ab86efba130a08724" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42233 } }
I20260809 15:39:51.489737 25603 sys_catalog.cc:564] T 00000000000000000000000000000000 P 1aa385c544c4407ab86efba130a08724 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 15:39:51.490935 25606 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:39:51.489073 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37667 (local address 127.0.0.1:42233)
0809 15:39:51.489172 (+ 99us) server_negotiation.cc:206] Beginning negotiation
0809 15:39:51.489176 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 15:39:51.489416 (+ 240us) server_negotiation.cc:407] Connection header received
0809 15:39:51.489554 (+ 138us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 15:39:51.489557 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 15:39:51.489590 (+ 33us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 15:39:51.489681 (+ 91us) server_negotiation.cc:226] Negotiated authn=SASL
0809 15:39:51.490384 (+ 703us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 15:39:51.490387 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 15:39:51.490390 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 15:39:51.490401 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0809 15:39:51.490426 (+ 25us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 15:39:51.490444 (+ 18us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 15:39:51.490540 (+ 96us) server_negotiation.cc:1035] Waiting for connection context
0809 15:39:51.490712 (+ 172us) server_negotiation.cc:299] Negotiation successful
0809 15:39:51.490761 (+ 49us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":37}
I20260809 15:39:51.491206 25615 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1aa385c544c4407ab86efba130a08724 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 1aa385c544c4407ab86efba130a08724. Latest consensus state: current_term: 1 leader_uuid: "1aa385c544c4407ab86efba130a08724" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1aa385c544c4407ab86efba130a08724" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42233 } } }
I20260809 15:39:51.491693 25615 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1aa385c544c4407ab86efba130a08724 [sys.catalog]: This master's current role is: LEADER
I20260809 15:39:51.490991 25614 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1aa385c544c4407ab86efba130a08724 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "1aa385c544c4407ab86efba130a08724" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1aa385c544c4407ab86efba130a08724" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42233 } } }
I20260809 15:39:51.492539 25614 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1aa385c544c4407ab86efba130a08724 [sys.catalog]: This master's current role is: LEADER
W20260809 15:39:51.494691 25630 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 1aa385c544c4407ab86efba130a08724: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260809 15:39:51.494813 25630 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260809 15:39:51.494892 25629 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 15:39:51.495613 25629 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 15:39:51.500614 25629 catalog_manager.cc:1347] Generated new cluster ID: 0a0967156ea64e2caa1550eeac8d09c0
I20260809 15:39:51.500679 25629 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 15:39:51.526086 25629 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 15:39:51.526443 25629 catalog_manager.cc:1504] Loading token signing keys...
I20260809 15:39:51.531219 25629 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 1aa385c544c4407ab86efba130a08724: Generated new TSK 0
I20260809 15:39:51.531984 25629 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260809 15:39:51.535419 3431 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskm314YR/build/debug/bin/kudu with pid 25528
2026-08-09T15:39:51Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/2 (1447 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/3
Loading random data
Initializing database '/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786289732823074-3431-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 15:39:51 dist-test-slave-2w3w krb5kdc[25637](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 15:39:51 dist-test-slave-2w3w krb5kdc[25637](info): set up 2 sockets
Aug 09 15:39:51 dist-test-slave-2w3w krb5kdc[25637](info): commencing operation
krb5kdc: starting...
W20260809 15:39:52.618680 3431 mini_kdc.cc:121] Time spent starting KDC: real 1.061s user 0.000s sys 0.008s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786289732823074-3431-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786289732823074-3431-0/krb5kdc/test-user.keytab.
Aug 09 15:39:52 dist-test-slave-2w3w krb5kdc[25637](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786289992, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T15:39:52Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T15:39:52Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786289732823074-3431-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786289732823074-3431-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260809 15:39:52.750159 3431 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskm314YR/build/debug/bin/kudu
/tmp/dist-test-taskm314YR/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786289732823074-3431-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786289732823074-3431-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786289732823074-3431-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786289732823074-3431-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:43063
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786289732823074-3431-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:36757
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:43063
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786289732823074-3431-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786289732823074-3431-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786289732823074-3431-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 15:39:52.862799 25650 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 15:39:52.863166 25650 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 15:39:52.863252 25650 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 15:39:52.867340 25650 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 15:39:52.867432 25650 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 15:39:52.867472 25650 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 15:39:52.867528 25650 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 15:39:52.867565 25650 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 15:39:52.872849 25650 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:36757
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786289732823074-3431-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786289732823074-3431-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:43063
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786289732823074-3431-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:43063
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786289732823074-3431-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.25650
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786289732823074-3431-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 09 Aug 2026 14:46:55 UTC on ccycloud.quasar-yswkmf.root.comops.site
build id 4576540
I20260809 15:39:52.873976 25650 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 15:39:52.875033 25650 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 15:39:52.883540 25660 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 15:39:52.883634 25656 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 15:39:52.883540 25658 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 15:39:52.883700 25650 server_base.cc:1034] running on GCE node
I20260809 15:39:52.884464 25650 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 15:39:52.885591 25650 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 15:39:52.886811 25650 hybrid_clock.cc:648] HybridClock initialized: now 1786289992886801 us; error 28 us; skew 500 ppm
Aug 09 15:39:52 dist-test-slave-2w3w krb5kdc[25637](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786289992, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 15:39:52.889763 25650 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260809 15:39:52.890867 25650 webserver.cc:460] Webserver started at http://127.0.0.1:33113/ using document root <none> and password file <none>
I20260809 15:39:52.891459 25650 fs_manager.cc:359] Metadata directory not provided
I20260809 15:39:52.891558 25650 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 15:39:52.891798 25650 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 15:39:52.893836 25650 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskm314YR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786289732823074-3431-0/minicluster-data/master-0/data/instance:
uuid: "b4c921a6697b42e1abd2f3fdeceed03d"
format_stamp: "Formatted at 2026-08-09 15:39:52 on dist-test-slave-2w3w"
server_key: "c6230401d834bac8e5be7288794f8730"
server_key_iv: "0598f0818ee91fd62c23c453451cc205"
server_key_version: "encryptionkey@0"
I20260809 15:39:52.894409 25650 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskm314YR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786289732823074-3431-0/minicluster-data/master-0/wal/instance:
uuid: "b4c921a6697b42e1abd2f3fdeceed03d"
format_stamp: "Formatted at 2026-08-09 15:39:52 on dist-test-slave-2w3w"
server_key: "c6230401d834bac8e5be7288794f8730"
server_key_iv: "0598f0818ee91fd62c23c453451cc205"
server_key_version: "encryptionkey@0"
I20260809 15:39:52.898213 25650 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.006s sys 0.000s
I20260809 15:39:52.900641 25667 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 15:39:52.901921 25650 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260809 15:39:52.902060 25650 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskm314YR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786289732823074-3431-0/minicluster-data/master-0/data,/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786289732823074-3431-0/minicluster-data/master-0/wal
uuid: "b4c921a6697b42e1abd2f3fdeceed03d"
format_stamp: "Formatted at 2026-08-09 15:39:52 on dist-test-slave-2w3w"
server_key: "c6230401d834bac8e5be7288794f8730"
server_key_iv: "0598f0818ee91fd62c23c453451cc205"
server_key_version: "encryptionkey@0"
I20260809 15:39:52.902175 25650 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskm314YR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786289732823074-3431-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskm314YR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786289732823074-3431-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskm314YR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786289732823074-3431-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 15:39:52.912956 25650 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 15:39:52.916214 25650 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 15:39:52.916422 25650 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 15:39:52.923687 25650 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:43063
I20260809 15:39:52.923713 25725 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:43063 every 8 connection(s)
I20260809 15:39:52.924618 25650 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskm314YR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786289732823074-3431-0/minicluster-data/master-0/data/info.pb
I20260809 15:39:52.927235 3431 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskm314YR/build/debug/bin/kudu as pid 25650
I20260809 15:39:52.927336 25726 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 15:39:52.927371 3431 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskm314YR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786289732823074-3431-0/minicluster-data/master-0/wal/instance
I20260809 15:39:52.927649 3431 external_mini_cluster.cc:1428] Setting key ec092e2bf21e90e2cf9458a25365ad1a
I20260809 15:39:52.934576 25726 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b4c921a6697b42e1abd2f3fdeceed03d: Bootstrap starting.
I20260809 15:39:52.936898 25726 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P b4c921a6697b42e1abd2f3fdeceed03d: Neither blocks nor log segments found. Creating new log.
Aug 09 15:39:52 dist-test-slave-2w3w krb5kdc[25637](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786289992, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.0.1@KRBTEST.COM
I20260809 15:39:52.937836 25726 log.cc:824] T 00000000000000000000000000000000 P b4c921a6697b42e1abd2f3fdeceed03d: Log is configured to *not* fsync() on all Append() calls
I20260809 15:39:52.940412 25726 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b4c921a6697b42e1abd2f3fdeceed03d: No bootstrap required, opened a new log
I20260809 15:39:52.943568 25729 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:39:52.929087 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58659 (local address 127.0.0.1:43063)
0809 15:39:52.929520 (+ 433us) server_negotiation.cc:206] Beginning negotiation
0809 15:39:52.929538 (+ 18us) server_negotiation.cc:399] Waiting for connection header
0809 15:39:52.929580 (+ 42us) server_negotiation.cc:407] Connection header received
0809 15:39:52.930788 (+ 1208us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 15:39:52.930813 (+ 25us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 15:39:52.931258 (+ 445us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 15:39:52.931642 (+ 384us) server_negotiation.cc:226] Negotiated authn=SASL
0809 15:39:52.932334 (+ 692us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:39:52.933845 (+ 1511us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:39:52.934733 (+ 888us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:39:52.935277 (+ 544us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:39:52.935459 (+ 182us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:39:52.938198 (+ 2739us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 15:39:52.938206 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 15:39:52.938227 (+ 21us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 15:39:52.938263 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0809 15:39:52.940303 (+ 2040us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 15:39:52.940797 (+ 494us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 15:39:52.940804 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 15:39:52.940809 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0809 15:39:52.940955 (+ 146us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 15:39:52.941229 (+ 274us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 15:39:52.941232 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 15:39:52.941234 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 15:39:52.941726 (+ 492us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 15:39:52.941910 (+ 184us) server_negotiation.cc:1035] Waiting for connection context
0809 15:39:52.942162 (+ 252us) server_negotiation.cc:299] Negotiation successful
0809 15:39:52.942481 (+ 319us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":284,"thread_start_us":119,"threads_started":1}
I20260809 15:39:52.945111 25726 raft_consensus.cc:348] T 00000000000000000000000000000000 P b4c921a6697b42e1abd2f3fdeceed03d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b4c921a6697b42e1abd2f3fdeceed03d" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43063 } }
I20260809 15:39:52.945367 25726 raft_consensus.cc:374] T 00000000000000000000000000000000 P b4c921a6697b42e1abd2f3fdeceed03d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 15:39:52.945502 25726 raft_consensus.cc:729] T 00000000000000000000000000000000 P b4c921a6697b42e1abd2f3fdeceed03d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b4c921a6697b42e1abd2f3fdeceed03d, State: Initialized, Role: FOLLOWER
I20260809 15:39:52.946004 25726 consensus_queue.cc:260] T 00000000000000000000000000000000 P b4c921a6697b42e1abd2f3fdeceed03d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b4c921a6697b42e1abd2f3fdeceed03d" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43063 } }
I20260809 15:39:52.946189 25726 raft_consensus.cc:388] T 00000000000000000000000000000000 P b4c921a6697b42e1abd2f3fdeceed03d [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 15:39:52.946260 25726 raft_consensus.cc:482] T 00000000000000000000000000000000 P b4c921a6697b42e1abd2f3fdeceed03d [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 15:39:52.946369 25726 raft_consensus.cc:3037] T 00000000000000000000000000000000 P b4c921a6697b42e1abd2f3fdeceed03d [term 0 FOLLOWER]: Advancing to term 1
I20260809 15:39:52.946645 3431 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260809 15:39:52.947470 25726 raft_consensus.cc:504] T 00000000000000000000000000000000 P b4c921a6697b42e1abd2f3fdeceed03d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b4c921a6697b42e1abd2f3fdeceed03d" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43063 } }
I20260809 15:39:52.947814 25726 leader_election.cc:302] T 00000000000000000000000000000000 P b4c921a6697b42e1abd2f3fdeceed03d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: b4c921a6697b42e1abd2f3fdeceed03d; no voters:
I20260809 15:39:52.948153 25726 leader_election.cc:288] T 00000000000000000000000000000000 P b4c921a6697b42e1abd2f3fdeceed03d [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 15:39:52.948223 25734 raft_consensus.cc:2781] T 00000000000000000000000000000000 P b4c921a6697b42e1abd2f3fdeceed03d [term 1 FOLLOWER]: Leader election won for term 1
I20260809 15:39:52.948483 25734 raft_consensus.cc:686] T 00000000000000000000000000000000 P b4c921a6697b42e1abd2f3fdeceed03d [term 1 LEADER]: Becoming Leader. State: Replica: b4c921a6697b42e1abd2f3fdeceed03d, State: Running, Role: LEADER
I20260809 15:39:52.949380 25734 consensus_queue.cc:237] T 00000000000000000000000000000000 P b4c921a6697b42e1abd2f3fdeceed03d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b4c921a6697b42e1abd2f3fdeceed03d" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43063 } }
I20260809 15:39:52.949530 25726 sys_catalog.cc:564] T 00000000000000000000000000000000 P b4c921a6697b42e1abd2f3fdeceed03d [sys.catalog]: configured and running, proceeding with master startup.
I20260809 15:39:52.950765 25734 sys_catalog.cc:455] T 00000000000000000000000000000000 P b4c921a6697b42e1abd2f3fdeceed03d [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "b4c921a6697b42e1abd2f3fdeceed03d" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b4c921a6697b42e1abd2f3fdeceed03d" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43063 } } }
I20260809 15:39:52.950882 25734 sys_catalog.cc:458] T 00000000000000000000000000000000 P b4c921a6697b42e1abd2f3fdeceed03d [sys.catalog]: This master's current role is: LEADER
I20260809 15:39:52.951258 25736 sys_catalog.cc:455] T 00000000000000000000000000000000 P b4c921a6697b42e1abd2f3fdeceed03d [sys.catalog]: SysCatalogTable state changed. Reason: New leader b4c921a6697b42e1abd2f3fdeceed03d. Latest consensus state: current_term: 1 leader_uuid: "b4c921a6697b42e1abd2f3fdeceed03d" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b4c921a6697b42e1abd2f3fdeceed03d" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43063 } } }
I20260809 15:39:52.951370 25736 sys_catalog.cc:458] T 00000000000000000000000000000000 P b4c921a6697b42e1abd2f3fdeceed03d [sys.catalog]: This master's current role is: LEADER
I20260809 15:39:52.951691 25745 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 15:39:52.954900 25745 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 15:39:52.958216 25729 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:39:52.948891 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51533 (local address 127.0.0.1:43063)
0809 15:39:52.948991 (+ 100us) server_negotiation.cc:206] Beginning negotiation
0809 15:39:52.948995 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 15:39:52.949212 (+ 217us) server_negotiation.cc:407] Connection header received
0809 15:39:52.949336 (+ 124us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 15:39:52.949339 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 15:39:52.949385 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 15:39:52.949471 (+ 86us) server_negotiation.cc:226] Negotiated authn=SASL
0809 15:39:52.950312 (+ 841us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:39:52.951144 (+ 832us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:39:52.952434 (+ 1290us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:39:52.952692 (+ 258us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:39:52.952817 (+ 125us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:39:52.954512 (+ 1695us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 15:39:52.954516 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 15:39:52.954519 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 15:39:52.954543 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0809 15:39:52.956359 (+ 1816us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 15:39:52.956903 (+ 544us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 15:39:52.956907 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 15:39:52.956909 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 15:39:52.957004 (+ 95us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 15:39:52.957303 (+ 299us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 15:39:52.957308 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 15:39:52.957310 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 15:39:52.957502 (+ 192us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 15:39:52.957640 (+ 138us) server_negotiation.cc:1035] Waiting for connection context
0809 15:39:52.957930 (+ 290us) server_negotiation.cc:299] Negotiation successful
0809 15:39:52.958047 (+ 117us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":26}
I20260809 15:39:52.961511 25745 catalog_manager.cc:1347] Generated new cluster ID: 4cea456a26b9444a91ba54f8ca90bfba
I20260809 15:39:52.961581 25745 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 15:39:52.973848 25745 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 15:39:52.974931 25745 catalog_manager.cc:1504] Loading token signing keys...
I20260809 15:39:52.979167 25745 catalog_manager.cc:5868] T 00000000000000000000000000000000 P b4c921a6697b42e1abd2f3fdeceed03d: Generated new TSK 0
I20260809 15:39:52.979915 25745 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260809 15:39:52.983877 3431 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskm314YR/build/debug/bin/kudu with pid 25650
2026-08-09T15:39:52Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/3 (1449 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/4
Loading random data
Initializing database '/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786289732823074-3431-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 15:39:53 dist-test-slave-2w3w krb5kdc[25761](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 15:39:53 dist-test-slave-2w3w krb5kdc[25761](info): set up 2 sockets
Aug 09 15:39:53 dist-test-slave-2w3w krb5kdc[25761](info): commencing operation
krb5kdc: starting...
W20260809 15:39:54.093643 3431 mini_kdc.cc:121] Time spent starting KDC: real 1.087s user 0.003s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786289732823074-3431-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786289732823074-3431-0/krb5kdc/test-user.keytab.
Aug 09 15:39:54 dist-test-slave-2w3w krb5kdc[25761](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786289994, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T15:39:54Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T15:39:54Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786289732823074-3431-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786289732823074-3431-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260809 15:39:54.226754 3431 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskm314YR/build/debug/bin/kudu
/tmp/dist-test-taskm314YR/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786289732823074-3431-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786289732823074-3431-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786289732823074-3431-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786289732823074-3431-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:36553
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786289732823074-3431-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:38583
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:36553
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786289732823074-3431-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786289732823074-3431-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786289732823074-3431-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 15:39:54.342597 25775 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 15:39:54.342948 25775 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 15:39:54.343091 25775 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 15:39:54.347349 25775 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 15:39:54.347441 25775 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 15:39:54.347477 25775 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 15:39:54.347520 25775 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 15:39:54.347554 25775 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 15:39:54.353387 25775 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:38583
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786289732823074-3431-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786289732823074-3431-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:36553
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786289732823074-3431-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:36553
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786289732823074-3431-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.25775
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786289732823074-3431-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 09 Aug 2026 14:46:55 UTC on ccycloud.quasar-yswkmf.root.comops.site
build id 4576540
I20260809 15:39:54.354544 25775 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 15:39:54.355628 25775 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 15:39:54.364199 25783 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 15:39:54.364372 25775 server_base.cc:1034] running on GCE node
W20260809 15:39:54.364205 25786 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 15:39:54.364205 25782 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 15:39:54.365167 25775 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 15:39:54.366246 25775 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 15:39:54.367432 25775 hybrid_clock.cc:648] HybridClock initialized: now 1786289994367412 us; error 45 us; skew 500 ppm
Aug 09 15:39:54 dist-test-slave-2w3w krb5kdc[25761](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786289994, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 15:39:54.370253 25775 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260809 15:39:54.371265 25775 webserver.cc:460] Webserver started at http://127.0.0.1:33109/ using document root <none> and password file <none>
I20260809 15:39:54.371840 25775 fs_manager.cc:359] Metadata directory not provided
I20260809 15:39:54.371938 25775 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 15:39:54.372174 25775 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 15:39:54.374366 25775 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskm314YR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786289732823074-3431-0/minicluster-data/master-0/data/instance:
uuid: "2248614c380b4aa784078adc19abf167"
format_stamp: "Formatted at 2026-08-09 15:39:54 on dist-test-slave-2w3w"
server_key: "c5e1e6fb8bc2df8ddda8c88789cdbafc"
server_key_iv: "00431b75b4ba23e84a1ece85c6c4e002"
server_key_version: "encryptionkey@0"
I20260809 15:39:54.374951 25775 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskm314YR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786289732823074-3431-0/minicluster-data/master-0/wal/instance:
uuid: "2248614c380b4aa784078adc19abf167"
format_stamp: "Formatted at 2026-08-09 15:39:54 on dist-test-slave-2w3w"
server_key: "c5e1e6fb8bc2df8ddda8c88789cdbafc"
server_key_iv: "00431b75b4ba23e84a1ece85c6c4e002"
server_key_version: "encryptionkey@0"
I20260809 15:39:54.378662 25775 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.003s sys 0.000s
I20260809 15:39:54.381160 25793 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 15:39:54.382355 25775 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.000s sys 0.002s
I20260809 15:39:54.382508 25775 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskm314YR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786289732823074-3431-0/minicluster-data/master-0/data,/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786289732823074-3431-0/minicluster-data/master-0/wal
uuid: "2248614c380b4aa784078adc19abf167"
format_stamp: "Formatted at 2026-08-09 15:39:54 on dist-test-slave-2w3w"
server_key: "c5e1e6fb8bc2df8ddda8c88789cdbafc"
server_key_iv: "00431b75b4ba23e84a1ece85c6c4e002"
server_key_version: "encryptionkey@0"
I20260809 15:39:54.382630 25775 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskm314YR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786289732823074-3431-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskm314YR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786289732823074-3431-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskm314YR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786289732823074-3431-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 15:39:54.407604 25775 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 15:39:54.411194 25775 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 15:39:54.411428 25775 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 15:39:54.418959 25775 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:36553
I20260809 15:39:54.419018 25853 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:36553 every 8 connection(s)
I20260809 15:39:54.419970 25775 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskm314YR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786289732823074-3431-0/minicluster-data/master-0/data/info.pb
I20260809 15:39:54.422690 25854 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 15:39:54.424547 3431 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskm314YR/build/debug/bin/kudu as pid 25775
I20260809 15:39:54.424695 3431 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskm314YR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786289732823074-3431-0/minicluster-data/master-0/wal/instance
I20260809 15:39:54.424955 3431 external_mini_cluster.cc:1428] Setting key efcbccd1a1e8f5a7f782e2ada3e790d6
I20260809 15:39:54.429917 25854 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2248614c380b4aa784078adc19abf167: Bootstrap starting.
I20260809 15:39:54.432715 25854 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 2248614c380b4aa784078adc19abf167: Neither blocks nor log segments found. Creating new log.
I20260809 15:39:54.433692 25854 log.cc:824] T 00000000000000000000000000000000 P 2248614c380b4aa784078adc19abf167: Log is configured to *not* fsync() on all Append() calls
I20260809 15:39:54.435374 25857 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:39:54.426432 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53209 (local address 127.0.0.1:36553)
0809 15:39:54.426881 (+ 449us) server_negotiation.cc:206] Beginning negotiation
0809 15:39:54.426900 (+ 19us) server_negotiation.cc:399] Waiting for connection header
0809 15:39:54.426944 (+ 44us) server_negotiation.cc:407] Connection header received
0809 15:39:54.427862 (+ 918us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 15:39:54.427886 (+ 24us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 15:39:54.428343 (+ 457us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 15:39:54.428731 (+ 388us) server_negotiation.cc:226] Negotiated authn=SASL
0809 15:39:54.429191 (+ 460us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:39:54.430650 (+ 1459us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:39:54.431641 (+ 991us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:39:54.432324 (+ 683us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:39:54.432493 (+ 169us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:39:54.433357 (+ 864us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 15:39:54.433364 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 15:39:54.433380 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 15:39:54.433908 (+ 528us) server_negotiation.cc:922] calling sasl_server_start()
0809 15:39:54.433990 (+ 82us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 15:39:54.434022 (+ 32us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 15:39:54.434133 (+ 111us) server_negotiation.cc:1035] Waiting for connection context
0809 15:39:54.434303 (+ 170us) server_negotiation.cc:299] Negotiation successful
0809 15:39:54.434527 (+ 224us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":299,"thread_start_us":121,"threads_started":1}
I20260809 15:39:54.435977 25854 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2248614c380b4aa784078adc19abf167: No bootstrap required, opened a new log
I20260809 15:39:54.438359 3431 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260809 15:39:54.440835 25854 raft_consensus.cc:348] T 00000000000000000000000000000000 P 2248614c380b4aa784078adc19abf167 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2248614c380b4aa784078adc19abf167" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36553 } }
I20260809 15:39:54.441165 25854 raft_consensus.cc:374] T 00000000000000000000000000000000 P 2248614c380b4aa784078adc19abf167 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 15:39:54.441241 25854 raft_consensus.cc:729] T 00000000000000000000000000000000 P 2248614c380b4aa784078adc19abf167 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2248614c380b4aa784078adc19abf167, State: Initialized, Role: FOLLOWER
I20260809 15:39:54.441735 25854 consensus_queue.cc:260] T 00000000000000000000000000000000 P 2248614c380b4aa784078adc19abf167 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2248614c380b4aa784078adc19abf167" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36553 } }
I20260809 15:39:54.441900 25854 raft_consensus.cc:388] T 00000000000000000000000000000000 P 2248614c380b4aa784078adc19abf167 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 15:39:54.441967 25854 raft_consensus.cc:482] T 00000000000000000000000000000000 P 2248614c380b4aa784078adc19abf167 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 15:39:54.442068 25854 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 2248614c380b4aa784078adc19abf167 [term 0 FOLLOWER]: Advancing to term 1
I20260809 15:39:54.443296 25854 raft_consensus.cc:504] T 00000000000000000000000000000000 P 2248614c380b4aa784078adc19abf167 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2248614c380b4aa784078adc19abf167" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36553 } }
I20260809 15:39:54.443675 25854 leader_election.cc:302] T 00000000000000000000000000000000 P 2248614c380b4aa784078adc19abf167 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 2248614c380b4aa784078adc19abf167; no voters:
I20260809 15:39:54.444074 25854 leader_election.cc:288] T 00000000000000000000000000000000 P 2248614c380b4aa784078adc19abf167 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 15:39:54.444192 25864 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 2248614c380b4aa784078adc19abf167 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 15:39:54.444434 25864 raft_consensus.cc:686] T 00000000000000000000000000000000 P 2248614c380b4aa784078adc19abf167 [term 1 LEADER]: Becoming Leader. State: Replica: 2248614c380b4aa784078adc19abf167, State: Running, Role: LEADER
I20260809 15:39:54.444777 25864 consensus_queue.cc:237] T 00000000000000000000000000000000 P 2248614c380b4aa784078adc19abf167 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2248614c380b4aa784078adc19abf167" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36553 } }
I20260809 15:39:54.445381 25857 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:39:54.440846 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50447 (local address 127.0.0.1:36553)
0809 15:39:54.440983 (+ 137us) server_negotiation.cc:206] Beginning negotiation
0809 15:39:54.440988 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 15:39:54.441146 (+ 158us) server_negotiation.cc:407] Connection header received
0809 15:39:54.441312 (+ 166us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 15:39:54.441316 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 15:39:54.441367 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 15:39:54.441459 (+ 92us) server_negotiation.cc:226] Negotiated authn=SASL
0809 15:39:54.441766 (+ 307us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:39:54.442526 (+ 760us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:39:54.443420 (+ 894us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:39:54.443766 (+ 346us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:39:54.443889 (+ 123us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:39:54.444899 (+ 1010us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 15:39:54.444902 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 15:39:54.444905 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 15:39:54.444918 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0809 15:39:54.444946 (+ 28us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 15:39:54.444959 (+ 13us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 15:39:54.445037 (+ 78us) server_negotiation.cc:1035] Waiting for connection context
0809 15:39:54.445199 (+ 162us) server_negotiation.cc:299] Negotiation successful
0809 15:39:54.445254 (+ 55us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":55}
I20260809 15:39:54.445576 25854 sys_catalog.cc:564] T 00000000000000000000000000000000 P 2248614c380b4aa784078adc19abf167 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 15:39:54.446892 25865 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2248614c380b4aa784078adc19abf167 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 2248614c380b4aa784078adc19abf167. Latest consensus state: current_term: 1 leader_uuid: "2248614c380b4aa784078adc19abf167" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2248614c380b4aa784078adc19abf167" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36553 } } }
I20260809 15:39:54.447055 25865 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2248614c380b4aa784078adc19abf167 [sys.catalog]: This master's current role is: LEADER
I20260809 15:39:54.447392 25866 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2248614c380b4aa784078adc19abf167 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "2248614c380b4aa784078adc19abf167" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2248614c380b4aa784078adc19abf167" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36553 } } }
I20260809 15:39:54.447474 25874 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 15:39:54.447510 25866 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2248614c380b4aa784078adc19abf167 [sys.catalog]: This master's current role is: LEADER
I20260809 15:39:54.450729 25874 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 15:39:54.456300 25874 catalog_manager.cc:1347] Generated new cluster ID: f01208f2a4ba4c849044eba13bec7613
I20260809 15:39:54.456394 25874 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 15:39:54.472553 25874 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 15:39:54.473996 25874 catalog_manager.cc:1504] Loading token signing keys...
I20260809 15:39:54.484583 25874 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 2248614c380b4aa784078adc19abf167: Generated new TSK 0
I20260809 15:39:54.485332 25874 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260809 15:39:54.496196 3431 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskm314YR/build/debug/bin/kudu with pid 25775
2026-08-09T15:39:54Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/4 (1511 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/5
Loading random data
Initializing database '/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786289732823074-3431-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 15:39:54 dist-test-slave-2w3w krb5kdc[25887](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 15:39:54 dist-test-slave-2w3w krb5kdc[25887](info): set up 2 sockets
Aug 09 15:39:54 dist-test-slave-2w3w krb5kdc[25887](info): commencing operation
krb5kdc: starting...
W20260809 15:39:55.585327 3431 mini_kdc.cc:121] Time spent starting KDC: real 1.067s user 0.000s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786289732823074-3431-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786289732823074-3431-0/krb5kdc/test-user.keytab.
Aug 09 15:39:55 dist-test-slave-2w3w krb5kdc[25887](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786289995, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T15:39:55Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T15:39:55Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786289732823074-3431-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786289732823074-3431-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260809 15:39:55.714182 3431 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskm314YR/build/debug/bin/kudu
/tmp/dist-test-taskm314YR/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786289732823074-3431-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786289732823074-3431-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786289732823074-3431-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786289732823074-3431-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:34595
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786289732823074-3431-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:43101
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:34595
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786289732823074-3431-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786289732823074-3431-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786289732823074-3431-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 15:39:55.825042 25902 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 15:39:55.825428 25902 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 15:39:55.825503 25902 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 15:39:55.829810 25902 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 15:39:55.829909 25902 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 15:39:55.829948 25902 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 15:39:55.829984 25902 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 15:39:55.830021 25902 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 15:39:55.835304 25902 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:43101
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786289732823074-3431-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786289732823074-3431-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:34595
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786289732823074-3431-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:34595
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786289732823074-3431-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.25902
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786289732823074-3431-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 09 Aug 2026 14:46:55 UTC on ccycloud.quasar-yswkmf.root.comops.site
build id 4576540
I20260809 15:39:55.836280 25902 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 15:39:55.837252 25902 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 15:39:55.845607 25910 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 15:39:55.845599 25908 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 15:39:55.845599 25907 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 15:39:55.845847 25902 server_base.cc:1034] running on GCE node
I20260809 15:39:55.846467 25902 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 15:39:55.847508 25902 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 15:39:55.848696 25902 hybrid_clock.cc:648] HybridClock initialized: now 1786289995848678 us; error 29 us; skew 500 ppm
Aug 09 15:39:55 dist-test-slave-2w3w krb5kdc[25887](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786289995, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 15:39:55.855994 25902 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260809 15:39:55.856803 25902 webserver.cc:460] Webserver started at http://127.0.0.1:46293/ using document root <none> and password file <none>
I20260809 15:39:55.857329 25902 fs_manager.cc:359] Metadata directory not provided
I20260809 15:39:55.857393 25902 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 15:39:55.857548 25902 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 15:39:55.859539 25902 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskm314YR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786289732823074-3431-0/minicluster-data/master-0/data/instance:
uuid: "9e469f1a16de4d77bad1fa6039203350"
format_stamp: "Formatted at 2026-08-09 15:39:55 on dist-test-slave-2w3w"
server_key: "13204408a448b4c98f31bacb74e02db6"
server_key_iv: "5f8070cbf7e781577776804a43541391"
server_key_version: "encryptionkey@0"
I20260809 15:39:55.860064 25902 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskm314YR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786289732823074-3431-0/minicluster-data/master-0/wal/instance:
uuid: "9e469f1a16de4d77bad1fa6039203350"
format_stamp: "Formatted at 2026-08-09 15:39:55 on dist-test-slave-2w3w"
server_key: "13204408a448b4c98f31bacb74e02db6"
server_key_iv: "5f8070cbf7e781577776804a43541391"
server_key_version: "encryptionkey@0"
I20260809 15:39:55.863572 25902 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.003s sys 0.001s
I20260809 15:39:55.865762 25918 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 15:39:55.866890 25902 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260809 15:39:55.867063 25902 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskm314YR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786289732823074-3431-0/minicluster-data/master-0/data,/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786289732823074-3431-0/minicluster-data/master-0/wal
uuid: "9e469f1a16de4d77bad1fa6039203350"
format_stamp: "Formatted at 2026-08-09 15:39:55 on dist-test-slave-2w3w"
server_key: "13204408a448b4c98f31bacb74e02db6"
server_key_iv: "5f8070cbf7e781577776804a43541391"
server_key_version: "encryptionkey@0"
I20260809 15:39:55.867167 25902 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskm314YR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786289732823074-3431-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskm314YR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786289732823074-3431-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskm314YR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786289732823074-3431-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 15:39:55.871520 25902 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 15:39:55.874122 25902 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 15:39:55.874302 25902 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 15:39:55.881403 25902 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:34595
I20260809 15:39:55.881421 25975 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:34595 every 8 connection(s)
I20260809 15:39:55.882267 25902 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskm314YR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786289732823074-3431-0/minicluster-data/master-0/data/info.pb
I20260809 15:39:55.884940 25977 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 15:39:55.890110 3431 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskm314YR/build/debug/bin/kudu as pid 25902
I20260809 15:39:55.890251 3431 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskm314YR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786289732823074-3431-0/minicluster-data/master-0/wal/instance
I20260809 15:39:55.890516 3431 external_mini_cluster.cc:1428] Setting key 390a6e228e629ee3a51b90e15eca079c
I20260809 15:39:55.890522 25977 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 9e469f1a16de4d77bad1fa6039203350: Bootstrap starting.
I20260809 15:39:55.893116 25977 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 9e469f1a16de4d77bad1fa6039203350: Neither blocks nor log segments found. Creating new log.
I20260809 15:39:55.893860 25977 log.cc:824] T 00000000000000000000000000000000 P 9e469f1a16de4d77bad1fa6039203350: Log is configured to *not* fsync() on all Append() calls
I20260809 15:39:55.895876 25980 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:39:55.891944 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48769 (local address 127.0.0.1:34595)
0809 15:39:55.892408 (+ 464us) server_negotiation.cc:206] Beginning negotiation
0809 15:39:55.892422 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0809 15:39:55.892456 (+ 34us) server_negotiation.cc:407] Connection header received
0809 15:39:55.893095 (+ 639us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 15:39:55.893111 (+ 16us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 15:39:55.893449 (+ 338us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 15:39:55.893777 (+ 328us) server_negotiation.cc:226] Negotiated authn=SASL
0809 15:39:55.894608 (+ 831us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 15:39:55.894615 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 15:39:55.894629 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 15:39:55.894643 (+ 14us) server_negotiation.cc:922] calling sasl_server_start()
0809 15:39:55.894727 (+ 84us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 15:39:55.894751 (+ 24us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 15:39:55.894838 (+ 87us) server_negotiation.cc:1035] Waiting for connection context
0809 15:39:55.895035 (+ 197us) server_negotiation.cc:299] Negotiation successful
0809 15:39:55.895209 (+ 174us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":305,"thread_start_us":133,"threads_started":1}
I20260809 15:39:55.896040 25977 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 9e469f1a16de4d77bad1fa6039203350: No bootstrap required, opened a new log
I20260809 15:39:55.898635 3431 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260809 15:39:55.900521 25977 raft_consensus.cc:348] T 00000000000000000000000000000000 P 9e469f1a16de4d77bad1fa6039203350 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9e469f1a16de4d77bad1fa6039203350" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34595 } }
I20260809 15:39:55.900799 25977 raft_consensus.cc:374] T 00000000000000000000000000000000 P 9e469f1a16de4d77bad1fa6039203350 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 15:39:55.900887 25977 raft_consensus.cc:729] T 00000000000000000000000000000000 P 9e469f1a16de4d77bad1fa6039203350 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9e469f1a16de4d77bad1fa6039203350, State: Initialized, Role: FOLLOWER
I20260809 15:39:55.901410 25977 consensus_queue.cc:260] T 00000000000000000000000000000000 P 9e469f1a16de4d77bad1fa6039203350 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9e469f1a16de4d77bad1fa6039203350" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34595 } }
I20260809 15:39:55.901626 25977 raft_consensus.cc:388] T 00000000000000000000000000000000 P 9e469f1a16de4d77bad1fa6039203350 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 15:39:55.901700 25977 raft_consensus.cc:482] T 00000000000000000000000000000000 P 9e469f1a16de4d77bad1fa6039203350 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 15:39:55.901803 25977 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 9e469f1a16de4d77bad1fa6039203350 [term 0 FOLLOWER]: Advancing to term 1
I20260809 15:39:55.902863 25980 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:39:55.901060 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54773 (local address 127.0.0.1:34595)
0809 15:39:55.901191 (+ 131us) server_negotiation.cc:206] Beginning negotiation
0809 15:39:55.901195 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 15:39:55.901403 (+ 208us) server_negotiation.cc:407] Connection header received
0809 15:39:55.901519 (+ 116us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 15:39:55.901521 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 15:39:55.901547 (+ 26us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 15:39:55.901643 (+ 96us) server_negotiation.cc:226] Negotiated authn=SASL
0809 15:39:55.902457 (+ 814us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 15:39:55.902460 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 15:39:55.902462 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 15:39:55.902473 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0809 15:39:55.902497 (+ 24us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 15:39:55.902510 (+ 13us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 15:39:55.902604 (+ 94us) server_negotiation.cc:1035] Waiting for connection context
0809 15:39:55.902703 (+ 99us) server_negotiation.cc:299] Negotiation successful
0809 15:39:55.902738 (+ 35us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":47}
I20260809 15:39:55.902926 25977 raft_consensus.cc:504] T 00000000000000000000000000000000 P 9e469f1a16de4d77bad1fa6039203350 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9e469f1a16de4d77bad1fa6039203350" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34595 } }
I20260809 15:39:55.903304 25977 leader_election.cc:302] T 00000000000000000000000000000000 P 9e469f1a16de4d77bad1fa6039203350 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 9e469f1a16de4d77bad1fa6039203350; no voters:
I20260809 15:39:55.903627 25977 leader_election.cc:288] T 00000000000000000000000000000000 P 9e469f1a16de4d77bad1fa6039203350 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 15:39:55.903755 25989 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 9e469f1a16de4d77bad1fa6039203350 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 15:39:55.903968 25989 raft_consensus.cc:686] T 00000000000000000000000000000000 P 9e469f1a16de4d77bad1fa6039203350 [term 1 LEADER]: Becoming Leader. State: Replica: 9e469f1a16de4d77bad1fa6039203350, State: Running, Role: LEADER
I20260809 15:39:55.904246 25989 consensus_queue.cc:237] T 00000000000000000000000000000000 P 9e469f1a16de4d77bad1fa6039203350 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9e469f1a16de4d77bad1fa6039203350" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34595 } }
I20260809 15:39:55.904826 25977 sys_catalog.cc:564] T 00000000000000000000000000000000 P 9e469f1a16de4d77bad1fa6039203350 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 15:39:55.905814 25990 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9e469f1a16de4d77bad1fa6039203350 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "9e469f1a16de4d77bad1fa6039203350" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9e469f1a16de4d77bad1fa6039203350" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34595 } } }
I20260809 15:39:55.905947 25990 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9e469f1a16de4d77bad1fa6039203350 [sys.catalog]: This master's current role is: LEADER
I20260809 15:39:55.906304 25999 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 15:39:55.906260 25991 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9e469f1a16de4d77bad1fa6039203350 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 9e469f1a16de4d77bad1fa6039203350. Latest consensus state: current_term: 1 leader_uuid: "9e469f1a16de4d77bad1fa6039203350" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9e469f1a16de4d77bad1fa6039203350" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34595 } } }
I20260809 15:39:55.906419 25991 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9e469f1a16de4d77bad1fa6039203350 [sys.catalog]: This master's current role is: LEADER
I20260809 15:39:55.909446 25999 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 15:39:55.916144 25999 catalog_manager.cc:1347] Generated new cluster ID: dd767aebaeaa44269512d68894db6be0
I20260809 15:39:55.916211 25999 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 15:39:55.943619 25999 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 15:39:55.943966 25999 catalog_manager.cc:1504] Loading token signing keys...
I20260809 15:39:55.949230 25999 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 9e469f1a16de4d77bad1fa6039203350: Generated new TSK 0
I20260809 15:39:55.949934 25999 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260809 15:39:55.965283 3431 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskm314YR/build/debug/bin/kudu with pid 25902
2026-08-09T15:39:55Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/5 (1468 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/6
Loading random data
Initializing database '/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786289732823074-3431-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 15:39:56 dist-test-slave-2w3w krb5kdc[26011](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 15:39:56 dist-test-slave-2w3w krb5kdc[26011](info): set up 2 sockets
Aug 09 15:39:56 dist-test-slave-2w3w krb5kdc[26011](info): commencing operation
krb5kdc: starting...
W20260809 15:39:57.045195 3431 mini_kdc.cc:121] Time spent starting KDC: real 1.057s user 0.003s sys 0.004s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786289732823074-3431-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786289732823074-3431-0/krb5kdc/test-user.keytab.
Aug 09 15:39:57 dist-test-slave-2w3w krb5kdc[26011](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786289997, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T15:39:57Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T15:39:57Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786289732823074-3431-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786289732823074-3431-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260809 15:39:57.174479 3431 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskm314YR/build/debug/bin/kudu
/tmp/dist-test-taskm314YR/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786289732823074-3431-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786289732823074-3431-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786289732823074-3431-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786289732823074-3431-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:36559
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786289732823074-3431-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:45487
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:36559
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786289732823074-3431-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786289732823074-3431-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786289732823074-3431-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 15:39:57.283435 26029 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 15:39:57.283735 26029 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 15:39:57.283810 26029 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 15:39:57.287830 26029 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 15:39:57.287911 26029 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 15:39:57.287948 26029 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 15:39:57.288014 26029 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 15:39:57.288053 26029 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 15:39:57.293236 26029 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:45487
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786289732823074-3431-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786289732823074-3431-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:36559
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786289732823074-3431-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:36559
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786289732823074-3431-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.26029
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786289732823074-3431-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 09 Aug 2026 14:46:55 UTC on ccycloud.quasar-yswkmf.root.comops.site
build id 4576540
I20260809 15:39:57.294279 26029 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 15:39:57.295346 26029 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 15:39:57.303419 26034 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 15:39:57.303450 26035 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 15:39:57.303419 26037 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 15:39:57.303632 26029 server_base.cc:1034] running on GCE node
I20260809 15:39:57.304451 26029 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 15:39:57.305511 26029 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 15:39:57.306661 26029 hybrid_clock.cc:648] HybridClock initialized: now 1786289997306646 us; error 36 us; skew 500 ppm
Aug 09 15:39:57 dist-test-slave-2w3w krb5kdc[26011](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786289997, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 15:39:57.309440 26029 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260809 15:39:57.310289 26029 webserver.cc:460] Webserver started at http://127.0.0.1:43463/ using document root <none> and password file <none>
I20260809 15:39:57.310791 26029 fs_manager.cc:359] Metadata directory not provided
I20260809 15:39:57.310848 26029 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 15:39:57.311082 26029 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 15:39:57.313087 26029 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskm314YR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786289732823074-3431-0/minicluster-data/master-0/data/instance:
uuid: "8ec2bc4d9b014aa09c4fb4e558639510"
format_stamp: "Formatted at 2026-08-09 15:39:57 on dist-test-slave-2w3w"
server_key: "5ed568455b68760389f3d43a4fbdc73c"
server_key_iv: "ab4401433df23f4b79350a1959cdcac5"
server_key_version: "encryptionkey@0"
I20260809 15:39:57.313602 26029 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskm314YR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786289732823074-3431-0/minicluster-data/master-0/wal/instance:
uuid: "8ec2bc4d9b014aa09c4fb4e558639510"
format_stamp: "Formatted at 2026-08-09 15:39:57 on dist-test-slave-2w3w"
server_key: "5ed568455b68760389f3d43a4fbdc73c"
server_key_iv: "ab4401433df23f4b79350a1959cdcac5"
server_key_version: "encryptionkey@0"
I20260809 15:39:57.317533 26029 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.001s sys 0.004s
I20260809 15:39:57.319861 26045 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 15:39:57.321045 26029 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260809 15:39:57.321193 26029 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskm314YR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786289732823074-3431-0/minicluster-data/master-0/data,/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786289732823074-3431-0/minicluster-data/master-0/wal
uuid: "8ec2bc4d9b014aa09c4fb4e558639510"
format_stamp: "Formatted at 2026-08-09 15:39:57 on dist-test-slave-2w3w"
server_key: "5ed568455b68760389f3d43a4fbdc73c"
server_key_iv: "ab4401433df23f4b79350a1959cdcac5"
server_key_version: "encryptionkey@0"
I20260809 15:39:57.321306 26029 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskm314YR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786289732823074-3431-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskm314YR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786289732823074-3431-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskm314YR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786289732823074-3431-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 15:39:57.332800 26029 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 15:39:57.335595 26029 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 15:39:57.335796 26029 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 15:39:57.342953 26029 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:36559
I20260809 15:39:57.342988 26101 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:36559 every 8 connection(s)
I20260809 15:39:57.344004 26029 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskm314YR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786289732823074-3431-0/minicluster-data/master-0/data/info.pb
I20260809 15:39:57.346897 26102 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 15:39:57.350944 3431 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskm314YR/build/debug/bin/kudu as pid 26029
I20260809 15:39:57.351145 3431 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskm314YR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786289732823074-3431-0/minicluster-data/master-0/wal/instance
I20260809 15:39:57.351447 3431 external_mini_cluster.cc:1428] Setting key 74ff426f71425c29a3d9fe106597ed16
I20260809 15:39:57.353046 26102 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8ec2bc4d9b014aa09c4fb4e558639510: Bootstrap starting.
I20260809 15:39:57.355396 26102 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 8ec2bc4d9b014aa09c4fb4e558639510: Neither blocks nor log segments found. Creating new log.
I20260809 15:39:57.356104 26102 log.cc:824] T 00000000000000000000000000000000 P 8ec2bc4d9b014aa09c4fb4e558639510: Log is configured to *not* fsync() on all Append() calls
I20260809 15:39:57.357861 26102 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8ec2bc4d9b014aa09c4fb4e558639510: No bootstrap required, opened a new log
Aug 09 15:39:57 dist-test-slave-2w3w krb5kdc[26011](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786289997, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.0.1@KRBTEST.COM
I20260809 15:39:57.361653 26102 raft_consensus.cc:348] T 00000000000000000000000000000000 P 8ec2bc4d9b014aa09c4fb4e558639510 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8ec2bc4d9b014aa09c4fb4e558639510" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36559 } }
I20260809 15:39:57.361918 26102 raft_consensus.cc:374] T 00000000000000000000000000000000 P 8ec2bc4d9b014aa09c4fb4e558639510 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 15:39:57.362032 26102 raft_consensus.cc:729] T 00000000000000000000000000000000 P 8ec2bc4d9b014aa09c4fb4e558639510 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8ec2bc4d9b014aa09c4fb4e558639510, State: Initialized, Role: FOLLOWER
I20260809 15:39:57.362552 26102 consensus_queue.cc:260] T 00000000000000000000000000000000 P 8ec2bc4d9b014aa09c4fb4e558639510 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8ec2bc4d9b014aa09c4fb4e558639510" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36559 } }
I20260809 15:39:57.362707 26102 raft_consensus.cc:388] T 00000000000000000000000000000000 P 8ec2bc4d9b014aa09c4fb4e558639510 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 15:39:57.362798 26102 raft_consensus.cc:482] T 00000000000000000000000000000000 P 8ec2bc4d9b014aa09c4fb4e558639510 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 15:39:57.362920 26102 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 8ec2bc4d9b014aa09c4fb4e558639510 [term 0 FOLLOWER]: Advancing to term 1
I20260809 15:39:57.364193 26102 raft_consensus.cc:504] T 00000000000000000000000000000000 P 8ec2bc4d9b014aa09c4fb4e558639510 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8ec2bc4d9b014aa09c4fb4e558639510" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36559 } }
I20260809 15:39:57.364573 26102 leader_election.cc:302] T 00000000000000000000000000000000 P 8ec2bc4d9b014aa09c4fb4e558639510 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 8ec2bc4d9b014aa09c4fb4e558639510; no voters:
I20260809 15:39:57.364923 26102 leader_election.cc:288] T 00000000000000000000000000000000 P 8ec2bc4d9b014aa09c4fb4e558639510 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 15:39:57.365023 26109 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 8ec2bc4d9b014aa09c4fb4e558639510 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 15:39:57.365341 26109 raft_consensus.cc:686] T 00000000000000000000000000000000 P 8ec2bc4d9b014aa09c4fb4e558639510 [term 1 LEADER]: Becoming Leader. State: Replica: 8ec2bc4d9b014aa09c4fb4e558639510, State: Running, Role: LEADER
I20260809 15:39:57.365710 26109 consensus_queue.cc:237] T 00000000000000000000000000000000 P 8ec2bc4d9b014aa09c4fb4e558639510 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8ec2bc4d9b014aa09c4fb4e558639510" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36559 } }
I20260809 15:39:57.366184 26102 sys_catalog.cc:564] T 00000000000000000000000000000000 P 8ec2bc4d9b014aa09c4fb4e558639510 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 15:39:57.366480 26107 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:39:57.352882 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.84:35621 (local address 127.0.0.1:36559)
0809 15:39:57.353345 (+ 463us) server_negotiation.cc:206] Beginning negotiation
0809 15:39:57.353360 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0809 15:39:57.353395 (+ 35us) server_negotiation.cc:407] Connection header received
0809 15:39:57.354101 (+ 706us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 15:39:57.354117 (+ 16us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 15:39:57.354427 (+ 310us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 15:39:57.354759 (+ 332us) server_negotiation.cc:226] Negotiated authn=SASL
0809 15:39:57.355699 (+ 940us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:39:57.356750 (+ 1051us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:39:57.357616 (+ 866us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:39:57.358140 (+ 524us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:39:57.358303 (+ 163us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:39:57.360977 (+ 2674us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 15:39:57.360982 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 15:39:57.360995 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 15:39:57.361024 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0809 15:39:57.362915 (+ 1891us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 15:39:57.363549 (+ 634us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 15:39:57.363554 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 15:39:57.363559 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0809 15:39:57.363672 (+ 113us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 15:39:57.364040 (+ 368us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 15:39:57.364042 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 15:39:57.364043 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 15:39:57.364490 (+ 447us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 15:39:57.364650 (+ 160us) server_negotiation.cc:1035] Waiting for connection context
0809 15:39:57.365080 (+ 430us) server_negotiation.cc:299] Negotiation successful
0809 15:39:57.365535 (+ 455us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":325,"thread_start_us":154,"threads_started":1}
I20260809 15:39:57.366905 26109 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8ec2bc4d9b014aa09c4fb4e558639510 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 8ec2bc4d9b014aa09c4fb4e558639510. Latest consensus state: current_term: 1 leader_uuid: "8ec2bc4d9b014aa09c4fb4e558639510" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8ec2bc4d9b014aa09c4fb4e558639510" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36559 } } }
I20260809 15:39:57.367668 26109 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8ec2bc4d9b014aa09c4fb4e558639510 [sys.catalog]: This master's current role is: LEADER
I20260809 15:39:57.367551 26110 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8ec2bc4d9b014aa09c4fb4e558639510 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "8ec2bc4d9b014aa09c4fb4e558639510" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8ec2bc4d9b014aa09c4fb4e558639510" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36559 } } }
I20260809 15:39:57.368000 26110 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8ec2bc4d9b014aa09c4fb4e558639510 [sys.catalog]: This master's current role is: LEADER
I20260809 15:39:57.368076 26119 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 15:39:57.370393 3431 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260809 15:39:57.371503 26119 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 15:39:57.377209 26119 catalog_manager.cc:1347] Generated new cluster ID: 37b1a2b42a6a457da270d6e0094d6581
I20260809 15:39:57.377333 26119 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 15:39:57.381218 26107 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:39:57.372324 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.84:47433 (local address 127.0.0.1:36559)
0809 15:39:57.372431 (+ 107us) server_negotiation.cc:206] Beginning negotiation
0809 15:39:57.372435 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 15:39:57.372761 (+ 326us) server_negotiation.cc:407] Connection header received
0809 15:39:57.372903 (+ 142us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 15:39:57.372906 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 15:39:57.372958 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 15:39:57.373060 (+ 102us) server_negotiation.cc:226] Negotiated authn=SASL
0809 15:39:57.373733 (+ 673us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:39:57.374525 (+ 792us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:39:57.375668 (+ 1143us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:39:57.376159 (+ 491us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:39:57.376272 (+ 113us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:39:57.377928 (+ 1656us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 15:39:57.377932 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 15:39:57.377934 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 15:39:57.377956 (+ 22us) server_negotiation.cc:922] calling sasl_server_start()
0809 15:39:57.379520 (+ 1564us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 15:39:57.379982 (+ 462us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 15:39:57.379985 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 15:39:57.379989 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 15:39:57.380063 (+ 74us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 15:39:57.380389 (+ 326us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 15:39:57.380392 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 15:39:57.380393 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 15:39:57.380554 (+ 161us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 15:39:57.380703 (+ 149us) server_negotiation.cc:1035] Waiting for connection context
0809 15:39:57.380967 (+ 264us) server_negotiation.cc:299] Negotiation successful
0809 15:39:57.381056 (+ 89us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":30}
I20260809 15:39:57.397688 26119 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 15:39:57.398648 26119 catalog_manager.cc:1504] Loading token signing keys...
I20260809 15:39:57.405735 26119 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 8ec2bc4d9b014aa09c4fb4e558639510: Generated new TSK 0
I20260809 15:39:57.406409 26119 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260809 15:39:57.415172 3431 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskm314YR/build/debug/bin/kudu with pid 26029
W20260809 15:39:57.421234 26009 connection.cc:535] client connection to 127.0.0.1:36559 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:36559): Connection reset by peer (error 104)
2026-08-09T15:39:57Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/6 (1448 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/7
Loading random data
Initializing database '/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786289732823074-3431-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 15:39:57 dist-test-slave-2w3w krb5kdc[26137](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 15:39:57 dist-test-slave-2w3w krb5kdc[26137](info): set up 2 sockets
Aug 09 15:39:57 dist-test-slave-2w3w krb5kdc[26137](info): commencing operation
krb5kdc: starting...
W20260809 15:39:58.564857 3431 mini_kdc.cc:121] Time spent starting KDC: real 1.129s user 0.003s sys 0.004s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786289732823074-3431-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786289732823074-3431-0/krb5kdc/test-user.keytab.
Aug 09 15:39:58 dist-test-slave-2w3w krb5kdc[26137](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786289998, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T15:39:58Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T15:39:58Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786289732823074-3431-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786289732823074-3431-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260809 15:39:58.689467 3431 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskm314YR/build/debug/bin/kudu
/tmp/dist-test-taskm314YR/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786289732823074-3431-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786289732823074-3431-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786289732823074-3431-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786289732823074-3431-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:39475
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786289732823074-3431-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:39445
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:39475
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786289732823074-3431-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786289732823074-3431-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786289732823074-3431-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 15:39:58.796790 26150 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 15:39:58.797048 26150 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 15:39:58.797120 26150 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 15:39:58.801057 26150 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 15:39:58.801151 26150 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 15:39:58.801189 26150 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 15:39:58.801242 26150 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 15:39:58.801281 26150 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 15:39:58.806344 26150 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:39445
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786289732823074-3431-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786289732823074-3431-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:39475
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786289732823074-3431-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:39475
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786289732823074-3431-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.26150
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786289732823074-3431-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 09 Aug 2026 14:46:55 UTC on ccycloud.quasar-yswkmf.root.comops.site
build id 4576540
I20260809 15:39:58.807317 26150 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 15:39:58.808276 26150 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260809 15:39:58.816597 26150 server_base.cc:1034] running on GCE node
W20260809 15:39:58.816538 26156 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 15:39:58.816540 26158 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 15:39:58.816771 26155 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 15:39:58.817515 26150 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 15:39:58.818506 26150 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 15:39:58.819703 26150 hybrid_clock.cc:648] HybridClock initialized: now 1786289998819683 us; error 30 us; skew 500 ppm
Aug 09 15:39:58 dist-test-slave-2w3w krb5kdc[26137](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786289998, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 15:39:58.822360 26150 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260809 15:39:58.823263 26150 webserver.cc:460] Webserver started at http://127.0.0.1:40111/ using document root <none> and password file <none>
I20260809 15:39:58.823773 26150 fs_manager.cc:359] Metadata directory not provided
I20260809 15:39:58.823863 26150 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 15:39:58.824084 26150 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 15:39:58.826025 26150 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskm314YR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786289732823074-3431-0/minicluster-data/master-0/data/instance:
uuid: "92823bfb4b614500ac3c9db4e70ecbf2"
format_stamp: "Formatted at 2026-08-09 15:39:58 on dist-test-slave-2w3w"
server_key: "196c0d3a8a79540f6e16d7670a4fefd5"
server_key_iv: "98e383bdc6ea044c3c58e79368af7e12"
server_key_version: "encryptionkey@0"
I20260809 15:39:58.826603 26150 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskm314YR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786289732823074-3431-0/minicluster-data/master-0/wal/instance:
uuid: "92823bfb4b614500ac3c9db4e70ecbf2"
format_stamp: "Formatted at 2026-08-09 15:39:58 on dist-test-slave-2w3w"
server_key: "196c0d3a8a79540f6e16d7670a4fefd5"
server_key_iv: "98e383bdc6ea044c3c58e79368af7e12"
server_key_version: "encryptionkey@0"
I20260809 15:39:58.830047 26150 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.006s sys 0.000s
I20260809 15:39:58.832410 26170 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 15:39:58.833513 26150 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.000s sys 0.001s
I20260809 15:39:58.833647 26150 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskm314YR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786289732823074-3431-0/minicluster-data/master-0/data,/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786289732823074-3431-0/minicluster-data/master-0/wal
uuid: "92823bfb4b614500ac3c9db4e70ecbf2"
format_stamp: "Formatted at 2026-08-09 15:39:58 on dist-test-slave-2w3w"
server_key: "196c0d3a8a79540f6e16d7670a4fefd5"
server_key_iv: "98e383bdc6ea044c3c58e79368af7e12"
server_key_version: "encryptionkey@0"
I20260809 15:39:58.833750 26150 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskm314YR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786289732823074-3431-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskm314YR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786289732823074-3431-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskm314YR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786289732823074-3431-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 15:39:58.866042 26150 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 15:39:58.868873 26150 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 15:39:58.869072 26150 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 15:39:58.876199 26150 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:39475
I20260809 15:39:58.876228 26225 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:39475 every 8 connection(s)
I20260809 15:39:58.877085 26150 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskm314YR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786289732823074-3431-0/minicluster-data/master-0/data/info.pb
I20260809 15:39:58.879601 26226 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 15:39:58.884951 26226 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 92823bfb4b614500ac3c9db4e70ecbf2: Bootstrap starting.
I20260809 15:39:58.885704 3431 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskm314YR/build/debug/bin/kudu as pid 26150
I20260809 15:39:58.885816 3431 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskm314YR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786289732823074-3431-0/minicluster-data/master-0/wal/instance
I20260809 15:39:58.886143 3431 external_mini_cluster.cc:1428] Setting key 33462710a0537e25443cfd4d2065c5ff
I20260809 15:39:58.887686 26226 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 92823bfb4b614500ac3c9db4e70ecbf2: Neither blocks nor log segments found. Creating new log.
I20260809 15:39:58.888406 26226 log.cc:824] T 00000000000000000000000000000000 P 92823bfb4b614500ac3c9db4e70ecbf2: Log is configured to *not* fsync() on all Append() calls
I20260809 15:39:58.890388 26226 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 92823bfb4b614500ac3c9db4e70ecbf2: No bootstrap required, opened a new log
I20260809 15:39:58.893806 26226 raft_consensus.cc:348] T 00000000000000000000000000000000 P 92823bfb4b614500ac3c9db4e70ecbf2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "92823bfb4b614500ac3c9db4e70ecbf2" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39475 } }
I20260809 15:39:58.894084 26226 raft_consensus.cc:374] T 00000000000000000000000000000000 P 92823bfb4b614500ac3c9db4e70ecbf2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 15:39:58.894177 26226 raft_consensus.cc:729] T 00000000000000000000000000000000 P 92823bfb4b614500ac3c9db4e70ecbf2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 92823bfb4b614500ac3c9db4e70ecbf2, State: Initialized, Role: FOLLOWER
I20260809 15:39:58.894589 26226 consensus_queue.cc:260] T 00000000000000000000000000000000 P 92823bfb4b614500ac3c9db4e70ecbf2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "92823bfb4b614500ac3c9db4e70ecbf2" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39475 } }
I20260809 15:39:58.894716 26229 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:39:58.887230 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.84:50767 (local address 127.0.0.1:39475)
0809 15:39:58.887635 (+ 405us) server_negotiation.cc:206] Beginning negotiation
0809 15:39:58.887651 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0809 15:39:58.887685 (+ 34us) server_negotiation.cc:407] Connection header received
0809 15:39:58.888328 (+ 643us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 15:39:58.888343 (+ 15us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 15:39:58.888715 (+ 372us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 15:39:58.889093 (+ 378us) server_negotiation.cc:226] Negotiated authn=SASL
0809 15:39:58.889456 (+ 363us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:39:58.890385 (+ 929us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:39:58.891356 (+ 971us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:39:58.891826 (+ 470us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:39:58.892000 (+ 174us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:39:58.892890 (+ 890us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 15:39:58.892895 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 15:39:58.892907 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 15:39:58.893267 (+ 360us) server_negotiation.cc:922] calling sasl_server_start()
0809 15:39:58.893335 (+ 68us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 15:39:58.893357 (+ 22us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 15:39:58.893506 (+ 149us) server_negotiation.cc:1035] Waiting for connection context
0809 15:39:58.893700 (+ 194us) server_negotiation.cc:299] Negotiation successful
0809 15:39:58.893829 (+ 129us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":275,"thread_start_us":117,"threads_started":1}
I20260809 15:39:58.894753 26226 raft_consensus.cc:388] T 00000000000000000000000000000000 P 92823bfb4b614500ac3c9db4e70ecbf2 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 15:39:58.894908 26226 raft_consensus.cc:482] T 00000000000000000000000000000000 P 92823bfb4b614500ac3c9db4e70ecbf2 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 15:39:58.895056 26226 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 92823bfb4b614500ac3c9db4e70ecbf2 [term 0 FOLLOWER]: Advancing to term 1
I20260809 15:39:58.896068 26226 raft_consensus.cc:504] T 00000000000000000000000000000000 P 92823bfb4b614500ac3c9db4e70ecbf2 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "92823bfb4b614500ac3c9db4e70ecbf2" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39475 } }
I20260809 15:39:58.896399 26226 leader_election.cc:302] T 00000000000000000000000000000000 P 92823bfb4b614500ac3c9db4e70ecbf2 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 92823bfb4b614500ac3c9db4e70ecbf2; no voters:
I20260809 15:39:58.896714 26226 leader_election.cc:288] T 00000000000000000000000000000000 P 92823bfb4b614500ac3c9db4e70ecbf2 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 15:39:58.897081 26232 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 92823bfb4b614500ac3c9db4e70ecbf2 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 15:39:58.897336 26232 raft_consensus.cc:686] T 00000000000000000000000000000000 P 92823bfb4b614500ac3c9db4e70ecbf2 [term 1 LEADER]: Becoming Leader. State: Replica: 92823bfb4b614500ac3c9db4e70ecbf2, State: Running, Role: LEADER
I20260809 15:39:58.897702 3431 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260809 15:39:58.898003 26226 sys_catalog.cc:564] T 00000000000000000000000000000000 P 92823bfb4b614500ac3c9db4e70ecbf2 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 15:39:58.899025 26232 consensus_queue.cc:237] T 00000000000000000000000000000000 P 92823bfb4b614500ac3c9db4e70ecbf2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "92823bfb4b614500ac3c9db4e70ecbf2" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39475 } }
I20260809 15:39:58.900617 26231 sys_catalog.cc:455] T 00000000000000000000000000000000 P 92823bfb4b614500ac3c9db4e70ecbf2 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 92823bfb4b614500ac3c9db4e70ecbf2. Latest consensus state: current_term: 1 leader_uuid: "92823bfb4b614500ac3c9db4e70ecbf2" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "92823bfb4b614500ac3c9db4e70ecbf2" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39475 } } }
I20260809 15:39:58.900951 26231 sys_catalog.cc:458] T 00000000000000000000000000000000 P 92823bfb4b614500ac3c9db4e70ecbf2 [sys.catalog]: This master's current role is: LEADER
I20260809 15:39:58.900704 26233 sys_catalog.cc:455] T 00000000000000000000000000000000 P 92823bfb4b614500ac3c9db4e70ecbf2 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "92823bfb4b614500ac3c9db4e70ecbf2" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "92823bfb4b614500ac3c9db4e70ecbf2" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39475 } } }
I20260809 15:39:58.901204 26233 sys_catalog.cc:458] T 00000000000000000000000000000000 P 92823bfb4b614500ac3c9db4e70ecbf2 [sys.catalog]: This master's current role is: LEADER
W20260809 15:39:58.903524 26252 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 92823bfb4b614500ac3c9db4e70ecbf2: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260809 15:39:58.903604 26252 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260809 15:39:58.903690 26253 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 15:39:58.904649 26253 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 15:39:58.904815 26229 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:39:58.899857 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.84:60439 (local address 127.0.0.1:39475)
0809 15:39:58.899954 (+ 97us) server_negotiation.cc:206] Beginning negotiation
0809 15:39:58.899959 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 15:39:58.900289 (+ 330us) server_negotiation.cc:407] Connection header received
0809 15:39:58.900459 (+ 170us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 15:39:58.900464 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 15:39:58.900518 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 15:39:58.900571 (+ 53us) server_negotiation.cc:226] Negotiated authn=SASL
0809 15:39:58.901002 (+ 431us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:39:58.901742 (+ 740us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:39:58.902918 (+ 1176us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:39:58.903290 (+ 372us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:39:58.903421 (+ 131us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:39:58.904305 (+ 884us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 15:39:58.904308 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 15:39:58.904311 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 15:39:58.904324 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0809 15:39:58.904352 (+ 28us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 15:39:58.904366 (+ 14us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 15:39:58.904471 (+ 105us) server_negotiation.cc:1035] Waiting for connection context
0809 15:39:58.904647 (+ 176us) server_negotiation.cc:299] Negotiation successful
0809 15:39:58.904682 (+ 35us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":41}
I20260809 15:39:58.910144 26253 catalog_manager.cc:1347] Generated new cluster ID: daa5d1c243134d15801dd2107d04508e
I20260809 15:39:58.910215 26253 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 15:39:58.926940 26253 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 15:39:58.928149 26253 catalog_manager.cc:1504] Loading token signing keys...
I20260809 15:39:58.940483 26253 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 92823bfb4b614500ac3c9db4e70ecbf2: Generated new TSK 0
I20260809 15:39:58.941196 26253 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260809 15:39:58.951539 3431 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskm314YR/build/debug/bin/kudu with pid 26150
W20260809 15:39:58.958738 26135 connection.cc:535] client connection to 127.0.0.1:39475 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:39475): Connection reset by peer (error 104)
2026-08-09T15:39:58Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/7 (1537 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/8
Loading random data
Initializing database '/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786289732823074-3431-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 15:39:59 dist-test-slave-2w3w krb5kdc[26259](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 15:39:59 dist-test-slave-2w3w krb5kdc[26259](info): set up 2 sockets
Aug 09 15:39:59 dist-test-slave-2w3w krb5kdc[26259](info): commencing operation
krb5kdc: starting...
W20260809 15:40:00.037097 3431 mini_kdc.cc:121] Time spent starting KDC: real 1.064s user 0.000s sys 0.008s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786289732823074-3431-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786289732823074-3431-0/krb5kdc/test-user.keytab.
Aug 09 15:40:00 dist-test-slave-2w3w krb5kdc[26259](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786290000, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T15:40:00Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T15:40:00Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786289732823074-3431-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786289732823074-3431-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260809 15:40:00.161775 3431 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskm314YR/build/debug/bin/kudu
/tmp/dist-test-taskm314YR/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786289732823074-3431-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786289732823074-3431-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786289732823074-3431-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786289732823074-3431-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:39465
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786289732823074-3431-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:36455
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:39465
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786289732823074-3431-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786289732823074-3431-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786289732823074-3431-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 15:40:00.273146 26273 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 15:40:00.273442 26273 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 15:40:00.273532 26273 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 15:40:00.277563 26273 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 15:40:00.277657 26273 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 15:40:00.277698 26273 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 15:40:00.277761 26273 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 15:40:00.277801 26273 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 15:40:00.283070 26273 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:36455
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786289732823074-3431-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786289732823074-3431-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:39465
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786289732823074-3431-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:39465
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786289732823074-3431-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.26273
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786289732823074-3431-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 09 Aug 2026 14:46:55 UTC on ccycloud.quasar-yswkmf.root.comops.site
build id 4576540
I20260809 15:40:00.284101 26273 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 15:40:00.285117 26273 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 15:40:00.293869 26283 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 15:40:00.293998 26273 server_base.cc:1034] running on GCE node
W20260809 15:40:00.293867 26280 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 15:40:00.293862 26281 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 15:40:00.294777 26273 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 15:40:00.295899 26273 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 15:40:00.297060 26273 hybrid_clock.cc:648] HybridClock initialized: now 1786290000297038 us; error 39 us; skew 500 ppm
Aug 09 15:40:00 dist-test-slave-2w3w krb5kdc[26259](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786290000, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 15:40:00.299890 26273 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260809 15:40:00.300819 26273 webserver.cc:460] Webserver started at http://127.0.0.1:36477/ using document root <none> and password file <none>
I20260809 15:40:00.301386 26273 fs_manager.cc:359] Metadata directory not provided
I20260809 15:40:00.301483 26273 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 15:40:00.301719 26273 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 15:40:00.303859 26273 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskm314YR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786289732823074-3431-0/minicluster-data/master-0/data/instance:
uuid: "d475bb839a924d16bed615a795491f20"
format_stamp: "Formatted at 2026-08-09 15:40:00 on dist-test-slave-2w3w"
server_key: "1ccbb558ad5ffd344367080b8193b8d2"
server_key_iv: "aba5b3fbdbe26b3edcf10ba1682bb6c2"
server_key_version: "encryptionkey@0"
I20260809 15:40:00.304447 26273 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskm314YR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786289732823074-3431-0/minicluster-data/master-0/wal/instance:
uuid: "d475bb839a924d16bed615a795491f20"
format_stamp: "Formatted at 2026-08-09 15:40:00 on dist-test-slave-2w3w"
server_key: "1ccbb558ad5ffd344367080b8193b8d2"
server_key_iv: "aba5b3fbdbe26b3edcf10ba1682bb6c2"
server_key_version: "encryptionkey@0"
I20260809 15:40:00.308184 26273 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.000s sys 0.004s
I20260809 15:40:00.310516 26290 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 15:40:00.311848 26273 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.001s
I20260809 15:40:00.311995 26273 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskm314YR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786289732823074-3431-0/minicluster-data/master-0/data,/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786289732823074-3431-0/minicluster-data/master-0/wal
uuid: "d475bb839a924d16bed615a795491f20"
format_stamp: "Formatted at 2026-08-09 15:40:00 on dist-test-slave-2w3w"
server_key: "1ccbb558ad5ffd344367080b8193b8d2"
server_key_iv: "aba5b3fbdbe26b3edcf10ba1682bb6c2"
server_key_version: "encryptionkey@0"
I20260809 15:40:00.312108 26273 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskm314YR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786289732823074-3431-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskm314YR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786289732823074-3431-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskm314YR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786289732823074-3431-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 15:40:00.316691 26273 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 15:40:00.319939 26273 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 15:40:00.320140 26273 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 15:40:00.327278 26273 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:39465
I20260809 15:40:00.327307 26352 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:39465 every 8 connection(s)
I20260809 15:40:00.328176 26273 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskm314YR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786289732823074-3431-0/minicluster-data/master-0/data/info.pb
I20260809 15:40:00.330713 26353 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 15:40:00.336166 26353 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d475bb839a924d16bed615a795491f20: Bootstrap starting.
I20260809 15:40:00.337934 3431 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskm314YR/build/debug/bin/kudu as pid 26273
I20260809 15:40:00.338088 3431 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskm314YR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786289732823074-3431-0/minicluster-data/master-0/wal/instance
I20260809 15:40:00.338388 3431 external_mini_cluster.cc:1428] Setting key 36e19f728775d71e694d2221abb992f8
I20260809 15:40:00.338629 26353 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P d475bb839a924d16bed615a795491f20: Neither blocks nor log segments found. Creating new log.
I20260809 15:40:00.339485 26353 log.cc:824] T 00000000000000000000000000000000 P d475bb839a924d16bed615a795491f20: Log is configured to *not* fsync() on all Append() calls
I20260809 15:40:00.341377 26353 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d475bb839a924d16bed615a795491f20: No bootstrap required, opened a new log
I20260809 15:40:00.343370 26356 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:40:00.339524 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.84:45359 (local address 127.0.0.1:39465)
0809 15:40:00.340004 (+ 480us) server_negotiation.cc:206] Beginning negotiation
0809 15:40:00.340023 (+ 19us) server_negotiation.cc:399] Waiting for connection header
0809 15:40:00.340067 (+ 44us) server_negotiation.cc:407] Connection header received
0809 15:40:00.340782 (+ 715us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 15:40:00.340799 (+ 17us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 15:40:00.341053 (+ 254us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 15:40:00.341373 (+ 320us) server_negotiation.cc:226] Negotiated authn=SASL
0809 15:40:00.342143 (+ 770us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 15:40:00.342148 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 15:40:00.342162 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 15:40:00.342180 (+ 18us) server_negotiation.cc:922] calling sasl_server_start()
0809 15:40:00.342231 (+ 51us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 15:40:00.342250 (+ 19us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 15:40:00.342367 (+ 117us) server_negotiation.cc:1035] Waiting for connection context
0809 15:40:00.342486 (+ 119us) server_negotiation.cc:299] Negotiation successful
0809 15:40:00.342638 (+ 152us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":323,"thread_start_us":135,"threads_started":1}
I20260809 15:40:00.345187 26353 raft_consensus.cc:348] T 00000000000000000000000000000000 P d475bb839a924d16bed615a795491f20 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d475bb839a924d16bed615a795491f20" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39465 } }
I20260809 15:40:00.345496 26353 raft_consensus.cc:374] T 00000000000000000000000000000000 P d475bb839a924d16bed615a795491f20 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 15:40:00.345563 26353 raft_consensus.cc:729] T 00000000000000000000000000000000 P d475bb839a924d16bed615a795491f20 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d475bb839a924d16bed615a795491f20, State: Initialized, Role: FOLLOWER
I20260809 15:40:00.346015 3431 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260809 15:40:00.346024 26353 consensus_queue.cc:260] T 00000000000000000000000000000000 P d475bb839a924d16bed615a795491f20 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d475bb839a924d16bed615a795491f20" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39465 } }
I20260809 15:40:00.346215 26353 raft_consensus.cc:388] T 00000000000000000000000000000000 P d475bb839a924d16bed615a795491f20 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 15:40:00.346309 26353 raft_consensus.cc:482] T 00000000000000000000000000000000 P d475bb839a924d16bed615a795491f20 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 15:40:00.346406 26353 raft_consensus.cc:3037] T 00000000000000000000000000000000 P d475bb839a924d16bed615a795491f20 [term 0 FOLLOWER]: Advancing to term 1
I20260809 15:40:00.347421 26353 raft_consensus.cc:504] T 00000000000000000000000000000000 P d475bb839a924d16bed615a795491f20 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d475bb839a924d16bed615a795491f20" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39465 } }
I20260809 15:40:00.347752 26353 leader_election.cc:302] T 00000000000000000000000000000000 P d475bb839a924d16bed615a795491f20 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: d475bb839a924d16bed615a795491f20; no voters:
I20260809 15:40:00.348045 26353 leader_election.cc:288] T 00000000000000000000000000000000 P d475bb839a924d16bed615a795491f20 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 15:40:00.348227 26362 raft_consensus.cc:2781] T 00000000000000000000000000000000 P d475bb839a924d16bed615a795491f20 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 15:40:00.348524 26362 raft_consensus.cc:686] T 00000000000000000000000000000000 P d475bb839a924d16bed615a795491f20 [term 1 LEADER]: Becoming Leader. State: Replica: d475bb839a924d16bed615a795491f20, State: Running, Role: LEADER
I20260809 15:40:00.348857 26362 consensus_queue.cc:237] T 00000000000000000000000000000000 P d475bb839a924d16bed615a795491f20 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d475bb839a924d16bed615a795491f20" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39465 } }
I20260809 15:40:00.349251 26353 sys_catalog.cc:564] T 00000000000000000000000000000000 P d475bb839a924d16bed615a795491f20 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 15:40:00.350534 26365 sys_catalog.cc:455] T 00000000000000000000000000000000 P d475bb839a924d16bed615a795491f20 [sys.catalog]: SysCatalogTable state changed. Reason: New leader d475bb839a924d16bed615a795491f20. Latest consensus state: current_term: 1 leader_uuid: "d475bb839a924d16bed615a795491f20" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d475bb839a924d16bed615a795491f20" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39465 } } }
I20260809 15:40:00.350935 26365 sys_catalog.cc:458] T 00000000000000000000000000000000 P d475bb839a924d16bed615a795491f20 [sys.catalog]: This master's current role is: LEADER
I20260809 15:40:00.350826 26363 sys_catalog.cc:455] T 00000000000000000000000000000000 P d475bb839a924d16bed615a795491f20 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "d475bb839a924d16bed615a795491f20" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d475bb839a924d16bed615a795491f20" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39465 } } }
I20260809 15:40:00.350720 26356 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:40:00.348169 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.84:36077 (local address 127.0.0.1:39465)
0809 15:40:00.348280 (+ 111us) server_negotiation.cc:206] Beginning negotiation
0809 15:40:00.348285 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 15:40:00.348831 (+ 546us) server_negotiation.cc:407] Connection header received
0809 15:40:00.349222 (+ 391us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 15:40:00.349225 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 15:40:00.349257 (+ 32us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 15:40:00.349304 (+ 47us) server_negotiation.cc:226] Negotiated authn=SASL
0809 15:40:00.350185 (+ 881us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 15:40:00.350191 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 15:40:00.350195 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 15:40:00.350210 (+ 15us) server_negotiation.cc:922] calling sasl_server_start()
0809 15:40:00.350241 (+ 31us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 15:40:00.350256 (+ 15us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 15:40:00.350324 (+ 68us) server_negotiation.cc:1035] Waiting for connection context
0809 15:40:00.350554 (+ 230us) server_negotiation.cc:299] Negotiation successful
0809 15:40:00.350598 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":41}
I20260809 15:40:00.351406 26363 sys_catalog.cc:458] T 00000000000000000000000000000000 P d475bb839a924d16bed615a795491f20 [sys.catalog]: This master's current role is: LEADER
W20260809 15:40:00.354084 26379 catalog_manager.cc:1558] T 00000000000000000000000000000000 P d475bb839a924d16bed615a795491f20: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260809 15:40:00.354161 26379 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260809 15:40:00.354239 26378 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 15:40:00.355028 26378 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 15:40:00.359786 26378 catalog_manager.cc:1347] Generated new cluster ID: 97582a3e8c214d879089d070976db538
I20260809 15:40:00.359844 26378 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 15:40:00.368579 26378 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 15:40:00.368881 26378 catalog_manager.cc:1504] Loading token signing keys...
I20260809 15:40:00.376972 26378 catalog_manager.cc:5868] T 00000000000000000000000000000000 P d475bb839a924d16bed615a795491f20: Generated new TSK 0
I20260809 15:40:00.377617 26378 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260809 15:40:00.384258 3431 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskm314YR/build/debug/bin/kudu with pid 26273
2026-08-09T15:40:00Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/8 (1433 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/9
Loading random data
Initializing database '/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786289732823074-3431-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 15:40:00 dist-test-slave-2w3w krb5kdc[26385](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 15:40:00 dist-test-slave-2w3w krb5kdc[26385](info): set up 2 sockets
Aug 09 15:40:00 dist-test-slave-2w3w krb5kdc[26385](info): commencing operation
krb5kdc: starting...
W20260809 15:40:01.473694 3431 mini_kdc.cc:121] Time spent starting KDC: real 1.067s user 0.000s sys 0.008s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786289732823074-3431-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786289732823074-3431-0/krb5kdc/test-user.keytab.
Aug 09 15:40:01 dist-test-slave-2w3w krb5kdc[26385](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786290001, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T15:40:01Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T15:40:01Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786289732823074-3431-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786289732823074-3431-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260809 15:40:01.602154 3431 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskm314YR/build/debug/bin/kudu
/tmp/dist-test-taskm314YR/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786289732823074-3431-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786289732823074-3431-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786289732823074-3431-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786289732823074-3431-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:35771
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786289732823074-3431-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:36729
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:35771
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786289732823074-3431-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786289732823074-3431-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786289732823074-3431-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 15:40:01.714949 26400 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 15:40:01.715233 26400 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 15:40:01.715274 26400 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 15:40:01.719308 26400 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 15:40:01.719360 26400 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 15:40:01.719378 26400 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 15:40:01.719394 26400 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 15:40:01.719449 26400 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 15:40:01.724625 26400 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:36729
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786289732823074-3431-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786289732823074-3431-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:35771
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786289732823074-3431-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:35771
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786289732823074-3431-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.26400
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786289732823074-3431-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 09 Aug 2026 14:46:55 UTC on ccycloud.quasar-yswkmf.root.comops.site
build id 4576540
I20260809 15:40:01.725649 26400 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 15:40:01.726665 26400 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 15:40:01.734927 26409 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 15:40:01.734948 26412 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 15:40:01.734918 26410 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 15:40:01.735246 26400 server_base.cc:1034] running on GCE node
I20260809 15:40:01.735891 26400 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 15:40:01.736898 26400 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 15:40:01.738060 26400 hybrid_clock.cc:648] HybridClock initialized: now 1786290001738038 us; error 34 us; skew 500 ppm
Aug 09 15:40:01 dist-test-slave-2w3w krb5kdc[26385](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786290001, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 15:40:01.744112 26400 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260809 15:40:01.745062 26400 webserver.cc:460] Webserver started at http://127.0.0.1:39201/ using document root <none> and password file <none>
I20260809 15:40:01.745628 26400 fs_manager.cc:359] Metadata directory not provided
I20260809 15:40:01.745692 26400 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 15:40:01.745939 26400 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 15:40:01.748015 26400 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskm314YR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786289732823074-3431-0/minicluster-data/master-0/data/instance:
uuid: "f77fba8ebc3d41ff8f985744c694f0d0"
format_stamp: "Formatted at 2026-08-09 15:40:01 on dist-test-slave-2w3w"
server_key: "dab7e8f5edfdc8781ded9e88a2c0323c"
server_key_iv: "ed1c1e78839829851339ea850ab49330"
server_key_version: "encryptionkey@0"
I20260809 15:40:01.748610 26400 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskm314YR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786289732823074-3431-0/minicluster-data/master-0/wal/instance:
uuid: "f77fba8ebc3d41ff8f985744c694f0d0"
format_stamp: "Formatted at 2026-08-09 15:40:01 on dist-test-slave-2w3w"
server_key: "dab7e8f5edfdc8781ded9e88a2c0323c"
server_key_iv: "ed1c1e78839829851339ea850ab49330"
server_key_version: "encryptionkey@0"
I20260809 15:40:01.752264 26400 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.004s sys 0.000s
I20260809 15:40:01.754562 26419 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 15:40:01.755820 26400 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260809 15:40:01.755966 26400 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskm314YR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786289732823074-3431-0/minicluster-data/master-0/data,/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786289732823074-3431-0/minicluster-data/master-0/wal
uuid: "f77fba8ebc3d41ff8f985744c694f0d0"
format_stamp: "Formatted at 2026-08-09 15:40:01 on dist-test-slave-2w3w"
server_key: "dab7e8f5edfdc8781ded9e88a2c0323c"
server_key_iv: "ed1c1e78839829851339ea850ab49330"
server_key_version: "encryptionkey@0"
I20260809 15:40:01.756081 26400 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskm314YR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786289732823074-3431-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskm314YR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786289732823074-3431-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskm314YR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786289732823074-3431-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 15:40:01.776836 26400 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 15:40:01.780213 26400 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 15:40:01.780436 26400 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 15:40:01.787925 26400 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:35771
I20260809 15:40:01.787953 26475 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:35771 every 8 connection(s)
I20260809 15:40:01.788898 26400 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskm314YR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786289732823074-3431-0/minicluster-data/master-0/data/info.pb
I20260809 15:40:01.791599 26476 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 15:40:01.797254 26476 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f77fba8ebc3d41ff8f985744c694f0d0: Bootstrap starting.
I20260809 15:40:01.799109 3431 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskm314YR/build/debug/bin/kudu as pid 26400
I20260809 15:40:01.799254 3431 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskm314YR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786289732823074-3431-0/minicluster-data/master-0/wal/instance
I20260809 15:40:01.799559 3431 external_mini_cluster.cc:1428] Setting key f09dc2dfc7d7e25237c7b4a288ea1816
I20260809 15:40:01.799993 26476 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P f77fba8ebc3d41ff8f985744c694f0d0: Neither blocks nor log segments found. Creating new log.
I20260809 15:40:01.800843 26476 log.cc:824] T 00000000000000000000000000000000 P f77fba8ebc3d41ff8f985744c694f0d0: Log is configured to *not* fsync() on all Append() calls
I20260809 15:40:01.802839 26476 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f77fba8ebc3d41ff8f985744c694f0d0: No bootstrap required, opened a new log
I20260809 15:40:01.806205 26476 raft_consensus.cc:348] T 00000000000000000000000000000000 P f77fba8ebc3d41ff8f985744c694f0d0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f77fba8ebc3d41ff8f985744c694f0d0" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35771 } }
I20260809 15:40:01.806452 26476 raft_consensus.cc:374] T 00000000000000000000000000000000 P f77fba8ebc3d41ff8f985744c694f0d0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 15:40:01.806558 26476 raft_consensus.cc:729] T 00000000000000000000000000000000 P f77fba8ebc3d41ff8f985744c694f0d0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f77fba8ebc3d41ff8f985744c694f0d0, State: Initialized, Role: FOLLOWER
I20260809 15:40:01.807128 26476 consensus_queue.cc:260] T 00000000000000000000000000000000 P f77fba8ebc3d41ff8f985744c694f0d0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f77fba8ebc3d41ff8f985744c694f0d0" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35771 } }
I20260809 15:40:01.807256 26476 raft_consensus.cc:388] T 00000000000000000000000000000000 P f77fba8ebc3d41ff8f985744c694f0d0 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 15:40:01.807348 26476 raft_consensus.cc:482] T 00000000000000000000000000000000 P f77fba8ebc3d41ff8f985744c694f0d0 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 15:40:01.807463 26476 raft_consensus.cc:3037] T 00000000000000000000000000000000 P f77fba8ebc3d41ff8f985744c694f0d0 [term 0 FOLLOWER]: Advancing to term 1
I20260809 15:40:01.808352 26476 raft_consensus.cc:504] T 00000000000000000000000000000000 P f77fba8ebc3d41ff8f985744c694f0d0 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f77fba8ebc3d41ff8f985744c694f0d0" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35771 } }
Aug 09 15:40:01 dist-test-slave-2w3w krb5kdc[26385](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786290001, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.0.1@KRBTEST.COM
I20260809 15:40:01.808691 26476 leader_election.cc:302] T 00000000000000000000000000000000 P f77fba8ebc3d41ff8f985744c694f0d0 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: f77fba8ebc3d41ff8f985744c694f0d0; no voters:
I20260809 15:40:01.809028 26476 leader_election.cc:288] T 00000000000000000000000000000000 P f77fba8ebc3d41ff8f985744c694f0d0 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 15:40:01.809104 26481 raft_consensus.cc:2781] T 00000000000000000000000000000000 P f77fba8ebc3d41ff8f985744c694f0d0 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 15:40:01.809315 26481 raft_consensus.cc:686] T 00000000000000000000000000000000 P f77fba8ebc3d41ff8f985744c694f0d0 [term 1 LEADER]: Becoming Leader. State: Replica: f77fba8ebc3d41ff8f985744c694f0d0, State: Running, Role: LEADER
I20260809 15:40:01.809954 26481 consensus_queue.cc:237] T 00000000000000000000000000000000 P f77fba8ebc3d41ff8f985744c694f0d0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f77fba8ebc3d41ff8f985744c694f0d0" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35771 } }
I20260809 15:40:01.810353 26476 sys_catalog.cc:564] T 00000000000000000000000000000000 P f77fba8ebc3d41ff8f985744c694f0d0 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 15:40:01.811724 26483 sys_catalog.cc:455] T 00000000000000000000000000000000 P f77fba8ebc3d41ff8f985744c694f0d0 [sys.catalog]: SysCatalogTable state changed. Reason: New leader f77fba8ebc3d41ff8f985744c694f0d0. Latest consensus state: current_term: 1 leader_uuid: "f77fba8ebc3d41ff8f985744c694f0d0" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f77fba8ebc3d41ff8f985744c694f0d0" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35771 } } }
I20260809 15:40:01.811848 26483 sys_catalog.cc:458] T 00000000000000000000000000000000 P f77fba8ebc3d41ff8f985744c694f0d0 [sys.catalog]: This master's current role is: LEADER
I20260809 15:40:01.812235 26482 sys_catalog.cc:455] T 00000000000000000000000000000000 P f77fba8ebc3d41ff8f985744c694f0d0 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "f77fba8ebc3d41ff8f985744c694f0d0" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f77fba8ebc3d41ff8f985744c694f0d0" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35771 } } }
I20260809 15:40:01.812405 26482 sys_catalog.cc:458] T 00000000000000000000000000000000 P f77fba8ebc3d41ff8f985744c694f0d0 [sys.catalog]: This master's current role is: LEADER
I20260809 15:40:01.814952 26479 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:40:01.800743 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.84:54237 (local address 127.0.0.1:35771)
0809 15:40:01.801212 (+ 469us) server_negotiation.cc:206] Beginning negotiation
0809 15:40:01.801236 (+ 24us) server_negotiation.cc:399] Waiting for connection header
0809 15:40:01.801282 (+ 46us) server_negotiation.cc:407] Connection header received
0809 15:40:01.802068 (+ 786us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 15:40:01.802087 (+ 19us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 15:40:01.802435 (+ 348us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 15:40:01.802780 (+ 345us) server_negotiation.cc:226] Negotiated authn=SASL
0809 15:40:01.803693 (+ 913us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:40:01.804679 (+ 986us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:40:01.805578 (+ 899us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:40:01.806108 (+ 530us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:40:01.806277 (+ 169us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:40:01.809140 (+ 2863us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 15:40:01.809148 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 15:40:01.809167 (+ 19us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 15:40:01.809201 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0809 15:40:01.811151 (+ 1950us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 15:40:01.811838 (+ 687us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 15:40:01.811844 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 15:40:01.811850 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0809 15:40:01.811993 (+ 143us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 15:40:01.812407 (+ 414us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 15:40:01.812410 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 15:40:01.812412 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 15:40:01.812941 (+ 529us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 15:40:01.813170 (+ 229us) server_negotiation.cc:1035] Waiting for connection context
0809 15:40:01.813691 (+ 521us) server_negotiation.cc:299] Negotiation successful
0809 15:40:01.813942 (+ 251us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":309,"thread_start_us":135,"threads_started":1}
W20260809 15:40:01.815572 26500 catalog_manager.cc:1558] T 00000000000000000000000000000000 P f77fba8ebc3d41ff8f985744c694f0d0: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260809 15:40:01.815657 26500 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260809 15:40:01.815744 26492 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 15:40:01.816576 26492 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 15:40:01.818503 3431 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260809 15:40:01.823715 26492 catalog_manager.cc:1347] Generated new cluster ID: 6d05d33cb76a40428f90ee9448e055b1
I20260809 15:40:01.823788 26492 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 15:40:01.830129 26479 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:40:01.821448 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.84:48291 (local address 127.0.0.1:35771)
0809 15:40:01.821573 (+ 125us) server_negotiation.cc:206] Beginning negotiation
0809 15:40:01.821579 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 15:40:01.821840 (+ 261us) server_negotiation.cc:407] Connection header received
0809 15:40:01.822008 (+ 168us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 15:40:01.822013 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 15:40:01.822065 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 15:40:01.822139 (+ 74us) server_negotiation.cc:226] Negotiated authn=SASL
0809 15:40:01.823135 (+ 996us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:40:01.824051 (+ 916us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:40:01.824899 (+ 848us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:40:01.825152 (+ 253us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:40:01.825270 (+ 118us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:40:01.826763 (+ 1493us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 15:40:01.826771 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 15:40:01.826773 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 15:40:01.826795 (+ 22us) server_negotiation.cc:922] calling sasl_server_start()
0809 15:40:01.828299 (+ 1504us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 15:40:01.828813 (+ 514us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 15:40:01.828823 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 15:40:01.828825 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 15:40:01.828899 (+ 74us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 15:40:01.829239 (+ 340us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 15:40:01.829242 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 15:40:01.829244 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 15:40:01.829420 (+ 176us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 15:40:01.829595 (+ 175us) server_negotiation.cc:1035] Waiting for connection context
0809 15:40:01.829871 (+ 276us) server_negotiation.cc:299] Negotiation successful
0809 15:40:01.829967 (+ 96us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":42}
I20260809 15:40:01.847694 26492 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 15:40:01.848657 26492 catalog_manager.cc:1504] Loading token signing keys...
I20260809 15:40:01.852846 26492 catalog_manager.cc:5868] T 00000000000000000000000000000000 P f77fba8ebc3d41ff8f985744c694f0d0: Generated new TSK 0
I20260809 15:40:01.853488 26492 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260809 15:40:01.861724 3431 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskm314YR/build/debug/bin/kudu with pid 26400
W20260809 15:40:01.870115 26383 connection.cc:535] client connection to 127.0.0.1:35771 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:35771): Connection reset by peer (error 104)
2026-08-09T15:40:01Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/9 (1477 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/10
Loading random data
Initializing database '/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786289732823074-3431-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 15:40:01 dist-test-slave-2w3w krb5kdc[26513](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 15:40:01 dist-test-slave-2w3w krb5kdc[26513](info): set up 2 sockets
Aug 09 15:40:01 dist-test-slave-2w3w krb5kdc[26513](info): commencing operation
krb5kdc: starting...
W20260809 15:40:02.953039 3431 mini_kdc.cc:121] Time spent starting KDC: real 1.068s user 0.000s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786289732823074-3431-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786289732823074-3431-0/krb5kdc/test-user.keytab.
Aug 09 15:40:03 dist-test-slave-2w3w krb5kdc[26513](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786290003, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T15:40:03Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T15:40:03Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786289732823074-3431-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786289732823074-3431-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260809 15:40:03.082651 3431 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskm314YR/build/debug/bin/kudu
/tmp/dist-test-taskm314YR/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786289732823074-3431-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786289732823074-3431-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786289732823074-3431-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786289732823074-3431-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:40929
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786289732823074-3431-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:46323
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:40929
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786289732823074-3431-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786289732823074-3431-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786289732823074-3431-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 15:40:03.190734 26526 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 15:40:03.191040 26526 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 15:40:03.191143 26526 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 15:40:03.195075 26526 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 15:40:03.195123 26526 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 15:40:03.195142 26526 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 15:40:03.195199 26526 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 15:40:03.195226 26526 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 15:40:03.200311 26526 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:46323
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786289732823074-3431-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786289732823074-3431-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:40929
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786289732823074-3431-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:40929
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786289732823074-3431-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.26526
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786289732823074-3431-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 09 Aug 2026 14:46:55 UTC on ccycloud.quasar-yswkmf.root.comops.site
build id 4576540
I20260809 15:40:03.201263 26526 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 15:40:03.202224 26526 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 15:40:03.210716 26535 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 15:40:03.210755 26531 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 15:40:03.210747 26526 server_base.cc:1034] running on GCE node
W20260809 15:40:03.210847 26532 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 15:40:03.211549 26526 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 15:40:03.212641 26526 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 15:40:03.213811 26526 hybrid_clock.cc:648] HybridClock initialized: now 1786290003213773 us; error 52 us; skew 500 ppm
Aug 09 15:40:03 dist-test-slave-2w3w krb5kdc[26513](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786290003, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 15:40:03.216645 26526 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260809 15:40:03.217597 26526 webserver.cc:460] Webserver started at http://127.0.0.1:35701/ using document root <none> and password file <none>
I20260809 15:40:03.218160 26526 fs_manager.cc:359] Metadata directory not provided
I20260809 15:40:03.218225 26526 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 15:40:03.218521 26526 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 15:40:03.220680 26526 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskm314YR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786289732823074-3431-0/minicluster-data/master-0/data/instance:
uuid: "8625d601f3e54f4ea607f54dbb22374a"
format_stamp: "Formatted at 2026-08-09 15:40:03 on dist-test-slave-2w3w"
server_key: "dd7df98ad38734b3fee07c3184df5a60"
server_key_iv: "ca195827c61f3a9cf84099a307294c1f"
server_key_version: "encryptionkey@0"
I20260809 15:40:03.221240 26526 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskm314YR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786289732823074-3431-0/minicluster-data/master-0/wal/instance:
uuid: "8625d601f3e54f4ea607f54dbb22374a"
format_stamp: "Formatted at 2026-08-09 15:40:03 on dist-test-slave-2w3w"
server_key: "dd7df98ad38734b3fee07c3184df5a60"
server_key_iv: "ca195827c61f3a9cf84099a307294c1f"
server_key_version: "encryptionkey@0"
I20260809 15:40:03.224925 26526 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.004s sys 0.001s
I20260809 15:40:03.227313 26543 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 15:40:03.228500 26526 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260809 15:40:03.228626 26526 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskm314YR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786289732823074-3431-0/minicluster-data/master-0/data,/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786289732823074-3431-0/minicluster-data/master-0/wal
uuid: "8625d601f3e54f4ea607f54dbb22374a"
format_stamp: "Formatted at 2026-08-09 15:40:03 on dist-test-slave-2w3w"
server_key: "dd7df98ad38734b3fee07c3184df5a60"
server_key_iv: "ca195827c61f3a9cf84099a307294c1f"
server_key_version: "encryptionkey@0"
I20260809 15:40:03.228720 26526 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskm314YR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786289732823074-3431-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskm314YR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786289732823074-3431-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskm314YR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786289732823074-3431-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 15:40:03.258949 26526 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 15:40:03.261875 26526 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 15:40:03.262076 26526 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 15:40:03.269112 26526 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:40929
I20260809 15:40:03.269140 26598 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:40929 every 8 connection(s)
I20260809 15:40:03.270016 26526 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskm314YR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786289732823074-3431-0/minicluster-data/master-0/data/info.pb
I20260809 15:40:03.272624 26599 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 15:40:03.277935 26599 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8625d601f3e54f4ea607f54dbb22374a: Bootstrap starting.
I20260809 15:40:03.278653 3431 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskm314YR/build/debug/bin/kudu as pid 26526
I20260809 15:40:03.278798 3431 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskm314YR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786289732823074-3431-0/minicluster-data/master-0/wal/instance
I20260809 15:40:03.279137 3431 external_mini_cluster.cc:1428] Setting key f757d3a0f9ad1e99d4ca561baef5704a
I20260809 15:40:03.280864 26599 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 8625d601f3e54f4ea607f54dbb22374a: Neither blocks nor log segments found. Creating new log.
I20260809 15:40:03.281584 26599 log.cc:824] T 00000000000000000000000000000000 P 8625d601f3e54f4ea607f54dbb22374a: Log is configured to *not* fsync() on all Append() calls
I20260809 15:40:03.283607 26599 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8625d601f3e54f4ea607f54dbb22374a: No bootstrap required, opened a new log
I20260809 15:40:03.287575 26599 raft_consensus.cc:348] T 00000000000000000000000000000000 P 8625d601f3e54f4ea607f54dbb22374a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8625d601f3e54f4ea607f54dbb22374a" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40929 } }
I20260809 15:40:03.287778 26599 raft_consensus.cc:374] T 00000000000000000000000000000000 P 8625d601f3e54f4ea607f54dbb22374a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 15:40:03.287837 26599 raft_consensus.cc:729] T 00000000000000000000000000000000 P 8625d601f3e54f4ea607f54dbb22374a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8625d601f3e54f4ea607f54dbb22374a, State: Initialized, Role: FOLLOWER
I20260809 15:40:03.288100 26604 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:40:03.280357 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.84:36109 (local address 127.0.0.1:40929)
0809 15:40:03.280761 (+ 404us) server_negotiation.cc:206] Beginning negotiation
0809 15:40:03.280774 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0809 15:40:03.280807 (+ 33us) server_negotiation.cc:407] Connection header received
0809 15:40:03.281477 (+ 670us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 15:40:03.281495 (+ 18us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 15:40:03.281858 (+ 363us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 15:40:03.282223 (+ 365us) server_negotiation.cc:226] Negotiated authn=SASL
0809 15:40:03.282591 (+ 368us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:40:03.283598 (+ 1007us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:40:03.284839 (+ 1241us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:40:03.285300 (+ 461us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:40:03.285501 (+ 201us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:40:03.286384 (+ 883us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 15:40:03.286390 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 15:40:03.286403 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 15:40:03.286756 (+ 353us) server_negotiation.cc:922] calling sasl_server_start()
0809 15:40:03.286806 (+ 50us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 15:40:03.286827 (+ 21us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 15:40:03.286996 (+ 169us) server_negotiation.cc:1035] Waiting for connection context
0809 15:40:03.287172 (+ 176us) server_negotiation.cc:299] Negotiation successful
0809 15:40:03.287300 (+ 128us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":273,"thread_start_us":119,"threads_started":1}
I20260809 15:40:03.288309 26599 consensus_queue.cc:260] T 00000000000000000000000000000000 P 8625d601f3e54f4ea607f54dbb22374a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8625d601f3e54f4ea607f54dbb22374a" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40929 } }
I20260809 15:40:03.288486 26599 raft_consensus.cc:388] T 00000000000000000000000000000000 P 8625d601f3e54f4ea607f54dbb22374a [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 15:40:03.288575 26599 raft_consensus.cc:482] T 00000000000000000000000000000000 P 8625d601f3e54f4ea607f54dbb22374a [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 15:40:03.288697 26599 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 8625d601f3e54f4ea607f54dbb22374a [term 0 FOLLOWER]: Advancing to term 1
I20260809 15:40:03.289741 26599 raft_consensus.cc:504] T 00000000000000000000000000000000 P 8625d601f3e54f4ea607f54dbb22374a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8625d601f3e54f4ea607f54dbb22374a" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40929 } }
I20260809 15:40:03.290103 26599 leader_election.cc:302] T 00000000000000000000000000000000 P 8625d601f3e54f4ea607f54dbb22374a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 8625d601f3e54f4ea607f54dbb22374a; no voters:
I20260809 15:40:03.290432 26599 leader_election.cc:288] T 00000000000000000000000000000000 P 8625d601f3e54f4ea607f54dbb22374a [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 15:40:03.290546 26606 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 8625d601f3e54f4ea607f54dbb22374a [term 1 FOLLOWER]: Leader election won for term 1
I20260809 15:40:03.290776 26606 raft_consensus.cc:686] T 00000000000000000000000000000000 P 8625d601f3e54f4ea607f54dbb22374a [term 1 LEADER]: Becoming Leader. State: Replica: 8625d601f3e54f4ea607f54dbb22374a, State: Running, Role: LEADER
I20260809 15:40:03.291244 3431 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260809 15:40:03.291455 26606 consensus_queue.cc:237] T 00000000000000000000000000000000 P 8625d601f3e54f4ea607f54dbb22374a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8625d601f3e54f4ea607f54dbb22374a" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40929 } }
I20260809 15:40:03.291733 26599 sys_catalog.cc:564] T 00000000000000000000000000000000 P 8625d601f3e54f4ea607f54dbb22374a [sys.catalog]: configured and running, proceeding with master startup.
I20260809 15:40:03.292639 26606 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8625d601f3e54f4ea607f54dbb22374a [sys.catalog]: SysCatalogTable state changed. Reason: New leader 8625d601f3e54f4ea607f54dbb22374a. Latest consensus state: current_term: 1 leader_uuid: "8625d601f3e54f4ea607f54dbb22374a" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8625d601f3e54f4ea607f54dbb22374a" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40929 } } }
I20260809 15:40:03.292879 26606 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8625d601f3e54f4ea607f54dbb22374a [sys.catalog]: This master's current role is: LEADER
I20260809 15:40:03.293056 26608 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8625d601f3e54f4ea607f54dbb22374a [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "8625d601f3e54f4ea607f54dbb22374a" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8625d601f3e54f4ea607f54dbb22374a" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40929 } } }
I20260809 15:40:03.293155 26608 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8625d601f3e54f4ea607f54dbb22374a [sys.catalog]: This master's current role is: LEADER
W20260809 15:40:03.296694 26627 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 8625d601f3e54f4ea607f54dbb22374a: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260809 15:40:03.296789 26627 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260809 15:40:03.296895 26620 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 15:40:03.297588 26620 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 15:40:03.299189 26604 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:40:03.293566 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.84:49809 (local address 127.0.0.1:40929)
0809 15:40:03.293681 (+ 115us) server_negotiation.cc:206] Beginning negotiation
0809 15:40:03.293685 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 15:40:03.294200 (+ 515us) server_negotiation.cc:407] Connection header received
0809 15:40:03.294436 (+ 236us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 15:40:03.294439 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 15:40:03.294484 (+ 45us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 15:40:03.294525 (+ 41us) server_negotiation.cc:226] Negotiated authn=SASL
0809 15:40:03.294989 (+ 464us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:40:03.295794 (+ 805us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:40:03.297009 (+ 1215us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:40:03.297369 (+ 360us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:40:03.297448 (+ 79us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:40:03.298575 (+ 1127us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 15:40:03.298579 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 15:40:03.298581 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 15:40:03.298595 (+ 14us) server_negotiation.cc:922] calling sasl_server_start()
0809 15:40:03.298623 (+ 28us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 15:40:03.298637 (+ 14us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 15:40:03.298701 (+ 64us) server_negotiation.cc:1035] Waiting for connection context
0809 15:40:03.298995 (+ 294us) server_negotiation.cc:299] Negotiation successful
0809 15:40:03.299033 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":47}
I20260809 15:40:03.302544 26620 catalog_manager.cc:1347] Generated new cluster ID: f700b815dc164e36a68b75174aa706a8
I20260809 15:40:03.302608 26620 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 15:40:03.317521 26620 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 15:40:03.318608 26620 catalog_manager.cc:1504] Loading token signing keys...
I20260809 15:40:03.324932 26620 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 8625d601f3e54f4ea607f54dbb22374a: Generated new TSK 0
I20260809 15:40:03.325556 26620 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260809 15:40:03.336154 3431 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskm314YR/build/debug/bin/kudu with pid 26526
W20260809 15:40:03.344203 26511 connection.cc:535] client connection to 127.0.0.1:40929 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:40929): Connection reset by peer (error 104)
2026-08-09T15:40:03Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/10 (1473 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/11
Loading random data
Initializing database '/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786289732823074-3431-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 15:40:03 dist-test-slave-2w3w krb5kdc[26633](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 15:40:03 dist-test-slave-2w3w krb5kdc[26633](info): set up 2 sockets
Aug 09 15:40:03 dist-test-slave-2w3w krb5kdc[26633](info): commencing operation
krb5kdc: starting...
W20260809 15:40:04.433254 3431 mini_kdc.cc:121] Time spent starting KDC: real 1.075s user 0.001s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786289732823074-3431-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786289732823074-3431-0/krb5kdc/test-user.keytab.
Aug 09 15:40:04 dist-test-slave-2w3w krb5kdc[26633](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786290004, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T15:40:04Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T15:40:04Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786289732823074-3431-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786289732823074-3431-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260809 15:40:04.562260 3431 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskm314YR/build/debug/bin/kudu
/tmp/dist-test-taskm314YR/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786289732823074-3431-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786289732823074-3431-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786289732823074-3431-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786289732823074-3431-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:42657
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786289732823074-3431-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:44541
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:42657
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786289732823074-3431-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786289732823074-3431-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786289732823074-3431-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 15:40:04.671072 26653 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 15:40:04.671340 26653 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 15:40:04.671434 26653 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 15:40:04.675348 26653 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 15:40:04.675424 26653 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 15:40:04.675463 26653 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 15:40:04.675515 26653 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 15:40:04.675555 26653 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 15:40:04.680562 26653 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:44541
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786289732823074-3431-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786289732823074-3431-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:42657
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786289732823074-3431-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:42657
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786289732823074-3431-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.26653
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786289732823074-3431-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 09 Aug 2026 14:46:55 UTC on ccycloud.quasar-yswkmf.root.comops.site
build id 4576540
I20260809 15:40:04.681499 26653 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 15:40:04.682425 26653 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 15:40:04.690402 26658 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 15:40:04.690412 26660 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 15:40:04.690464 26662 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 15:40:04.690551 26653 server_base.cc:1034] running on GCE node
I20260809 15:40:04.691205 26653 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 15:40:04.692198 26653 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 15:40:04.693382 26653 hybrid_clock.cc:648] HybridClock initialized: now 1786290004693363 us; error 30 us; skew 500 ppm
Aug 09 15:40:04 dist-test-slave-2w3w krb5kdc[26633](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786290004, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 15:40:04.695999 26653 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260809 15:40:04.696789 26653 webserver.cc:460] Webserver started at http://127.0.0.1:41513/ using document root <none> and password file <none>
I20260809 15:40:04.697252 26653 fs_manager.cc:359] Metadata directory not provided
I20260809 15:40:04.697319 26653 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 15:40:04.697515 26653 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 15:40:04.699448 26653 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskm314YR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786289732823074-3431-0/minicluster-data/master-0/data/instance:
uuid: "fabc3ade7c17476f9f55116cbe0eb47b"
format_stamp: "Formatted at 2026-08-09 15:40:04 on dist-test-slave-2w3w"
server_key: "776ed1ee73485cd7039682a91121515a"
server_key_iv: "3f164c56a368845780bbe8eae541ec03"
server_key_version: "encryptionkey@0"
I20260809 15:40:04.699975 26653 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskm314YR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786289732823074-3431-0/minicluster-data/master-0/wal/instance:
uuid: "fabc3ade7c17476f9f55116cbe0eb47b"
format_stamp: "Formatted at 2026-08-09 15:40:04 on dist-test-slave-2w3w"
server_key: "776ed1ee73485cd7039682a91121515a"
server_key_iv: "3f164c56a368845780bbe8eae541ec03"
server_key_version: "encryptionkey@0"
I20260809 15:40:04.703477 26653 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.003s sys 0.001s
I20260809 15:40:04.705694 26669 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 15:40:04.706808 26653 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260809 15:40:04.707018 26653 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskm314YR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786289732823074-3431-0/minicluster-data/master-0/data,/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786289732823074-3431-0/minicluster-data/master-0/wal
uuid: "fabc3ade7c17476f9f55116cbe0eb47b"
format_stamp: "Formatted at 2026-08-09 15:40:04 on dist-test-slave-2w3w"
server_key: "776ed1ee73485cd7039682a91121515a"
server_key_iv: "3f164c56a368845780bbe8eae541ec03"
server_key_version: "encryptionkey@0"
I20260809 15:40:04.707129 26653 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskm314YR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786289732823074-3431-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskm314YR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786289732823074-3431-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskm314YR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786289732823074-3431-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 15:40:04.712019 26653 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 15:40:04.714636 26653 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 15:40:04.714831 26653 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 15:40:04.721832 26653 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:42657
I20260809 15:40:04.721863 26727 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:42657 every 8 connection(s)
I20260809 15:40:04.722698 26653 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskm314YR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786289732823074-3431-0/minicluster-data/master-0/data/info.pb
I20260809 15:40:04.725204 26728 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 15:40:04.727906 3431 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskm314YR/build/debug/bin/kudu as pid 26653
I20260809 15:40:04.728048 3431 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskm314YR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786289732823074-3431-0/minicluster-data/master-0/wal/instance
I20260809 15:40:04.728305 3431 external_mini_cluster.cc:1428] Setting key 5d44fbc4596276fd29bca8833b0b7b70
I20260809 15:40:04.731019 26728 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P fabc3ade7c17476f9f55116cbe0eb47b: Bootstrap starting.
I20260809 15:40:04.733625 26728 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P fabc3ade7c17476f9f55116cbe0eb47b: Neither blocks nor log segments found. Creating new log.
I20260809 15:40:04.733673 26731 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:40:04.729624 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.84:48889 (local address 127.0.0.1:42657)
0809 15:40:04.730060 (+ 436us) server_negotiation.cc:206] Beginning negotiation
0809 15:40:04.730081 (+ 21us) server_negotiation.cc:399] Waiting for connection header
0809 15:40:04.730121 (+ 40us) server_negotiation.cc:407] Connection header received
0809 15:40:04.730916 (+ 795us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 15:40:04.730936 (+ 20us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 15:40:04.731308 (+ 372us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 15:40:04.731707 (+ 399us) server_negotiation.cc:226] Negotiated authn=SASL
0809 15:40:04.732370 (+ 663us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 15:40:04.732376 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 15:40:04.732390 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 15:40:04.732401 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0809 15:40:04.732457 (+ 56us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 15:40:04.732492 (+ 35us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 15:40:04.732625 (+ 133us) server_negotiation.cc:1035] Waiting for connection context
0809 15:40:04.732686 (+ 61us) server_negotiation.cc:299] Negotiation successful
0809 15:40:04.732826 (+ 140us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":272,"thread_start_us":124,"threads_started":1}
I20260809 15:40:04.734377 26728 log.cc:824] T 00000000000000000000000000000000 P fabc3ade7c17476f9f55116cbe0eb47b: Log is configured to *not* fsync() on all Append() calls
I20260809 15:40:04.736192 3431 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260809 15:40:04.736405 26728 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P fabc3ade7c17476f9f55116cbe0eb47b: No bootstrap required, opened a new log
I20260809 15:40:04.739792 26731 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:40:04.738117 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.84:56171 (local address 127.0.0.1:42657)
0809 15:40:04.738235 (+ 118us) server_negotiation.cc:206] Beginning negotiation
0809 15:40:04.738239 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 15:40:04.738422 (+ 183us) server_negotiation.cc:407] Connection header received
0809 15:40:04.738537 (+ 115us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 15:40:04.738540 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 15:40:04.738573 (+ 33us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 15:40:04.738651 (+ 78us) server_negotiation.cc:226] Negotiated authn=SASL
0809 15:40:04.739382 (+ 731us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 15:40:04.739385 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 15:40:04.739388 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 15:40:04.739399 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0809 15:40:04.739424 (+ 25us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 15:40:04.739440 (+ 16us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 15:40:04.739529 (+ 89us) server_negotiation.cc:1035] Waiting for connection context
0809 15:40:04.739632 (+ 103us) server_negotiation.cc:299] Negotiation successful
0809 15:40:04.739670 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":36}
I20260809 15:40:04.740353 26728 raft_consensus.cc:348] T 00000000000000000000000000000000 P fabc3ade7c17476f9f55116cbe0eb47b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fabc3ade7c17476f9f55116cbe0eb47b" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42657 } }
I20260809 15:40:04.740584 26728 raft_consensus.cc:374] T 00000000000000000000000000000000 P fabc3ade7c17476f9f55116cbe0eb47b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 15:40:04.740651 26728 raft_consensus.cc:729] T 00000000000000000000000000000000 P fabc3ade7c17476f9f55116cbe0eb47b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fabc3ade7c17476f9f55116cbe0eb47b, State: Initialized, Role: FOLLOWER
I20260809 15:40:04.741050 26728 consensus_queue.cc:260] T 00000000000000000000000000000000 P fabc3ade7c17476f9f55116cbe0eb47b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fabc3ade7c17476f9f55116cbe0eb47b" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42657 } }
I20260809 15:40:04.741173 26728 raft_consensus.cc:388] T 00000000000000000000000000000000 P fabc3ade7c17476f9f55116cbe0eb47b [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 15:40:04.741217 26728 raft_consensus.cc:482] T 00000000000000000000000000000000 P fabc3ade7c17476f9f55116cbe0eb47b [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 15:40:04.741283 26728 raft_consensus.cc:3037] T 00000000000000000000000000000000 P fabc3ade7c17476f9f55116cbe0eb47b [term 0 FOLLOWER]: Advancing to term 1
I20260809 15:40:04.742066 26728 raft_consensus.cc:504] T 00000000000000000000000000000000 P fabc3ade7c17476f9f55116cbe0eb47b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fabc3ade7c17476f9f55116cbe0eb47b" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42657 } }
I20260809 15:40:04.742338 26728 leader_election.cc:302] T 00000000000000000000000000000000 P fabc3ade7c17476f9f55116cbe0eb47b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: fabc3ade7c17476f9f55116cbe0eb47b; no voters:
I20260809 15:40:04.742563 26728 leader_election.cc:288] T 00000000000000000000000000000000 P fabc3ade7c17476f9f55116cbe0eb47b [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 15:40:04.742659 26738 raft_consensus.cc:2781] T 00000000000000000000000000000000 P fabc3ade7c17476f9f55116cbe0eb47b [term 1 FOLLOWER]: Leader election won for term 1
I20260809 15:40:04.742892 26738 raft_consensus.cc:686] T 00000000000000000000000000000000 P fabc3ade7c17476f9f55116cbe0eb47b [term 1 LEADER]: Becoming Leader. State: Replica: fabc3ade7c17476f9f55116cbe0eb47b, State: Running, Role: LEADER
I20260809 15:40:04.743242 26738 consensus_queue.cc:237] T 00000000000000000000000000000000 P fabc3ade7c17476f9f55116cbe0eb47b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fabc3ade7c17476f9f55116cbe0eb47b" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42657 } }
I20260809 15:40:04.743736 26728 sys_catalog.cc:564] T 00000000000000000000000000000000 P fabc3ade7c17476f9f55116cbe0eb47b [sys.catalog]: configured and running, proceeding with master startup.
I20260809 15:40:04.744822 26742 sys_catalog.cc:455] T 00000000000000000000000000000000 P fabc3ade7c17476f9f55116cbe0eb47b [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "fabc3ade7c17476f9f55116cbe0eb47b" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fabc3ade7c17476f9f55116cbe0eb47b" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42657 } } }
I20260809 15:40:04.744951 26742 sys_catalog.cc:458] T 00000000000000000000000000000000 P fabc3ade7c17476f9f55116cbe0eb47b [sys.catalog]: This master's current role is: LEADER
I20260809 15:40:04.745285 26743 sys_catalog.cc:455] T 00000000000000000000000000000000 P fabc3ade7c17476f9f55116cbe0eb47b [sys.catalog]: SysCatalogTable state changed. Reason: New leader fabc3ade7c17476f9f55116cbe0eb47b. Latest consensus state: current_term: 1 leader_uuid: "fabc3ade7c17476f9f55116cbe0eb47b" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fabc3ade7c17476f9f55116cbe0eb47b" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42657 } } }
I20260809 15:40:04.745397 26750 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 15:40:04.745428 26743 sys_catalog.cc:458] T 00000000000000000000000000000000 P fabc3ade7c17476f9f55116cbe0eb47b [sys.catalog]: This master's current role is: LEADER
I20260809 15:40:04.748409 26750 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 15:40:04.753762 26750 catalog_manager.cc:1347] Generated new cluster ID: 383fb9dacd5742b691dfd90ba9a4bd32
I20260809 15:40:04.753832 26750 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 15:40:04.775168 26750 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 15:40:04.775470 26750 catalog_manager.cc:1504] Loading token signing keys...
I20260809 15:40:04.783419 26750 catalog_manager.cc:5868] T 00000000000000000000000000000000 P fabc3ade7c17476f9f55116cbe0eb47b: Generated new TSK 0
I20260809 15:40:04.784029 26750 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260809 15:40:04.787771 3431 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskm314YR/build/debug/bin/kudu with pid 26653
2026-08-09T15:40:04Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/11 (1450 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/12
Loading random data
Initializing database '/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786289732823074-3431-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 15:40:04 dist-test-slave-2w3w krb5kdc[26764](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 15:40:04 dist-test-slave-2w3w krb5kdc[26764](info): set up 2 sockets
Aug 09 15:40:04 dist-test-slave-2w3w krb5kdc[26764](info): commencing operation
krb5kdc: starting...
W20260809 15:40:05.912173 3431 mini_kdc.cc:121] Time spent starting KDC: real 1.103s user 0.006s sys 0.001s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786289732823074-3431-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786289732823074-3431-0/krb5kdc/test-user.keytab.
Aug 09 15:40:05 dist-test-slave-2w3w krb5kdc[26764](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786290005, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T15:40:06Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T15:40:06Z Disabled control of system clock
WARNING: no policy specified for kudu/127.3.89.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.3.89.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.3.89.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786289732823074-3431-0/krb5kdc/kudu_127.3.89.254.keytab.
Entry for principal kudu/127.3.89.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786289732823074-3431-0/krb5kdc/kudu_127.3.89.254.keytab.
I20260809 15:40:06.038312 3431 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskm314YR/build/debug/bin/kudu
/tmp/dist-test-taskm314YR/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786289732823074-3431-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786289732823074-3431-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786289732823074-3431-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786289732823074-3431-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.3.89.254:42641
--webserver_interface=127.3.89.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786289732823074-3431-0/krb5kdc/kudu_127.3.89.254.keytab
--principal=kudu/127.3.89.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.3.89.212:33037
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.3.89.254:42641
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786289732823074-3431-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786289732823074-3431-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786289732823074-3431-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 15:40:06.146793 26778 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 15:40:06.147135 26778 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 15:40:06.147223 26778 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 15:40:06.151852 26778 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 15:40:06.151933 26778 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 15:40:06.151969 26778 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 15:40:06.152005 26778 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 15:40:06.152040 26778 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 15:40:06.157168 26778 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.3.89.212:33037
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786289732823074-3431-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786289732823074-3431-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.3.89.254:42641
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786289732823074-3431-0/krb5kdc/kudu_127.3.89.254.keytab
--principal=kudu/127.3.89.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.3.89.254:42641
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786289732823074-3431-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.3.89.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.26778
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786289732823074-3431-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 09 Aug 2026 14:46:55 UTC on ccycloud.quasar-yswkmf.root.comops.site
build id 4576540
I20260809 15:40:06.158129 26778 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 15:40:06.159108 26778 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 15:40:06.167116 26784 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 15:40:06.167209 26778 server_base.cc:1034] running on GCE node
W20260809 15:40:06.167115 26786 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 15:40:06.167135 26783 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 15:40:06.167969 26778 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 15:40:06.168994 26778 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 15:40:06.170176 26778 hybrid_clock.cc:648] HybridClock initialized: now 1786290006170158 us; error 32 us; skew 500 ppm
Aug 09 15:40:06 dist-test-slave-2w3w krb5kdc[26764](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786290006, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.89.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 15:40:06.172969 26778 init.cc:377] Logged in from keytab as kudu/127.3.89.254@KRBTEST.COM (short username kudu)
I20260809 15:40:06.173838 26778 webserver.cc:460] Webserver started at http://127.3.89.254:44031/ using document root <none> and password file <none>
I20260809 15:40:06.174371 26778 fs_manager.cc:359] Metadata directory not provided
I20260809 15:40:06.174459 26778 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 15:40:06.174677 26778 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 15:40:06.176685 26778 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskm314YR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786289732823074-3431-0/minicluster-data/master-0/data/instance:
uuid: "c5f7169ee54c4588ab91659c5ada193d"
format_stamp: "Formatted at 2026-08-09 15:40:06 on dist-test-slave-2w3w"
server_key: "5e9dde51804eb7e527c494f4ddace91f"
server_key_iv: "53824340bc8d0b1016571e2b29516d1c"
server_key_version: "encryptionkey@0"
I20260809 15:40:06.177233 26778 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskm314YR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786289732823074-3431-0/minicluster-data/master-0/wal/instance:
uuid: "c5f7169ee54c4588ab91659c5ada193d"
format_stamp: "Formatted at 2026-08-09 15:40:06 on dist-test-slave-2w3w"
server_key: "5e9dde51804eb7e527c494f4ddace91f"
server_key_iv: "53824340bc8d0b1016571e2b29516d1c"
server_key_version: "encryptionkey@0"
I20260809 15:40:06.180780 26778 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.005s sys 0.000s
I20260809 15:40:06.183045 26794 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 15:40:06.184170 26778 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260809 15:40:06.184305 26778 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskm314YR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786289732823074-3431-0/minicluster-data/master-0/data,/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786289732823074-3431-0/minicluster-data/master-0/wal
uuid: "c5f7169ee54c4588ab91659c5ada193d"
format_stamp: "Formatted at 2026-08-09 15:40:06 on dist-test-slave-2w3w"
server_key: "5e9dde51804eb7e527c494f4ddace91f"
server_key_iv: "53824340bc8d0b1016571e2b29516d1c"
server_key_version: "encryptionkey@0"
I20260809 15:40:06.184408 26778 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskm314YR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786289732823074-3431-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskm314YR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786289732823074-3431-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskm314YR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786289732823074-3431-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 15:40:06.194039 26778 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 15:40:06.197189 26778 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 15:40:06.197386 26778 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 15:40:06.204473 26778 rpc_server.cc:307] RPC server started. Bound to: 127.3.89.254:42641
I20260809 15:40:06.204502 26852 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.3.89.254:42641 every 8 connection(s)
I20260809 15:40:06.205379 26778 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskm314YR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786289732823074-3431-0/minicluster-data/master-0/data/info.pb
I20260809 15:40:06.208045 26853 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 15:40:06.213248 26853 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c5f7169ee54c4588ab91659c5ada193d: Bootstrap starting.
I20260809 15:40:06.214084 3431 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskm314YR/build/debug/bin/kudu as pid 26778
I20260809 15:40:06.214239 3431 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskm314YR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786289732823074-3431-0/minicluster-data/master-0/wal/instance
I20260809 15:40:06.214537 3431 external_mini_cluster.cc:1428] Setting key 74b7f47baa649dcf0deebedef786c335
I20260809 15:40:06.216107 26853 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P c5f7169ee54c4588ab91659c5ada193d: Neither blocks nor log segments found. Creating new log.
I20260809 15:40:06.216825 26853 log.cc:824] T 00000000000000000000000000000000 P c5f7169ee54c4588ab91659c5ada193d: Log is configured to *not* fsync() on all Append() calls
I20260809 15:40:06.218907 26853 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c5f7169ee54c4588ab91659c5ada193d: No bootstrap required, opened a new log
I20260809 15:40:06.223224 26853 raft_consensus.cc:348] T 00000000000000000000000000000000 P c5f7169ee54c4588ab91659c5ada193d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c5f7169ee54c4588ab91659c5ada193d" member_type: VOTER last_known_addr { host: "127.3.89.254" port: 42641 } }
I20260809 15:40:06.223425 26853 raft_consensus.cc:374] T 00000000000000000000000000000000 P c5f7169ee54c4588ab91659c5ada193d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 15:40:06.223459 26853 raft_consensus.cc:729] T 00000000000000000000000000000000 P c5f7169ee54c4588ab91659c5ada193d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c5f7169ee54c4588ab91659c5ada193d, State: Initialized, Role: FOLLOWER
Aug 09 15:40:06 dist-test-slave-2w3w krb5kdc[26764](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786290005, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.3.89.254@KRBTEST.COM
I20260809 15:40:06.223891 26853 consensus_queue.cc:260] T 00000000000000000000000000000000 P c5f7169ee54c4588ab91659c5ada193d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c5f7169ee54c4588ab91659c5ada193d" member_type: VOTER last_known_addr { host: "127.3.89.254" port: 42641 } }
I20260809 15:40:06.224036 26853 raft_consensus.cc:388] T 00000000000000000000000000000000 P c5f7169ee54c4588ab91659c5ada193d [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 15:40:06.224116 26853 raft_consensus.cc:482] T 00000000000000000000000000000000 P c5f7169ee54c4588ab91659c5ada193d [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 15:40:06.224228 26853 raft_consensus.cc:3037] T 00000000000000000000000000000000 P c5f7169ee54c4588ab91659c5ada193d [term 0 FOLLOWER]: Advancing to term 1
I20260809 15:40:06.225070 26853 raft_consensus.cc:504] T 00000000000000000000000000000000 P c5f7169ee54c4588ab91659c5ada193d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c5f7169ee54c4588ab91659c5ada193d" member_type: VOTER last_known_addr { host: "127.3.89.254" port: 42641 } }
I20260809 15:40:06.225383 26853 leader_election.cc:302] T 00000000000000000000000000000000 P c5f7169ee54c4588ab91659c5ada193d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: c5f7169ee54c4588ab91659c5ada193d; no voters:
I20260809 15:40:06.225666 26853 leader_election.cc:288] T 00000000000000000000000000000000 P c5f7169ee54c4588ab91659c5ada193d [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 15:40:06.225776 26858 raft_consensus.cc:2781] T 00000000000000000000000000000000 P c5f7169ee54c4588ab91659c5ada193d [term 1 FOLLOWER]: Leader election won for term 1
I20260809 15:40:06.225991 26858 raft_consensus.cc:686] T 00000000000000000000000000000000 P c5f7169ee54c4588ab91659c5ada193d [term 1 LEADER]: Becoming Leader. State: Replica: c5f7169ee54c4588ab91659c5ada193d, State: Running, Role: LEADER
I20260809 15:40:06.226351 26858 consensus_queue.cc:237] T 00000000000000000000000000000000 P c5f7169ee54c4588ab91659c5ada193d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c5f7169ee54c4588ab91659c5ada193d" member_type: VOTER last_known_addr { host: "127.3.89.254" port: 42641 } }
I20260809 15:40:06.226831 26853 sys_catalog.cc:564] T 00000000000000000000000000000000 P c5f7169ee54c4588ab91659c5ada193d [sys.catalog]: configured and running, proceeding with master startup.
I20260809 15:40:06.227988 26859 sys_catalog.cc:455] T 00000000000000000000000000000000 P c5f7169ee54c4588ab91659c5ada193d [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "c5f7169ee54c4588ab91659c5ada193d" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c5f7169ee54c4588ab91659c5ada193d" member_type: VOTER last_known_addr { host: "127.3.89.254" port: 42641 } } }
I20260809 15:40:06.228368 26859 sys_catalog.cc:458] T 00000000000000000000000000000000 P c5f7169ee54c4588ab91659c5ada193d [sys.catalog]: This master's current role is: LEADER
I20260809 15:40:06.228262 26861 sys_catalog.cc:455] T 00000000000000000000000000000000 P c5f7169ee54c4588ab91659c5ada193d [sys.catalog]: SysCatalogTable state changed. Reason: New leader c5f7169ee54c4588ab91659c5ada193d. Latest consensus state: current_term: 1 leader_uuid: "c5f7169ee54c4588ab91659c5ada193d" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c5f7169ee54c4588ab91659c5ada193d" member_type: VOTER last_known_addr { host: "127.3.89.254" port: 42641 } } }
I20260809 15:40:06.228703 26861 sys_catalog.cc:458] T 00000000000000000000000000000000 P c5f7169ee54c4588ab91659c5ada193d [sys.catalog]: This master's current role is: LEADER
I20260809 15:40:06.230365 26856 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:40:06.215652 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37455 (local address 127.3.89.254:42641)
0809 15:40:06.216065 (+ 413us) server_negotiation.cc:206] Beginning negotiation
0809 15:40:06.216079 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0809 15:40:06.216112 (+ 33us) server_negotiation.cc:407] Connection header received
0809 15:40:06.216779 (+ 667us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 15:40:06.216796 (+ 17us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 15:40:06.217160 (+ 364us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 15:40:06.217513 (+ 353us) server_negotiation.cc:226] Negotiated authn=SASL
0809 15:40:06.218433 (+ 920us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:40:06.219511 (+ 1078us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:40:06.220786 (+ 1275us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:40:06.221259 (+ 473us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:40:06.221391 (+ 132us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:40:06.224469 (+ 3078us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 15:40:06.224474 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 15:40:06.224485 (+ 11us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 15:40:06.224510 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0809 15:40:06.226651 (+ 2141us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 15:40:06.227290 (+ 639us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 15:40:06.227299 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 15:40:06.227308 (+ 9us) server_negotiation.cc:967] calling sasl_server_step()
0809 15:40:06.227453 (+ 145us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 15:40:06.227939 (+ 486us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 15:40:06.227944 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 15:40:06.227946 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 15:40:06.228477 (+ 531us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 15:40:06.228704 (+ 227us) server_negotiation.cc:1035] Waiting for connection context
0809 15:40:06.229471 (+ 767us) server_negotiation.cc:299] Negotiation successful
0809 15:40:06.229700 (+ 229us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":287,"thread_start_us":138,"threads_started":1}
W20260809 15:40:06.231984 26875 catalog_manager.cc:1558] T 00000000000000000000000000000000 P c5f7169ee54c4588ab91659c5ada193d: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260809 15:40:06.232118 26875 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260809 15:40:06.232203 26872 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 15:40:06.232906 26872 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 15:40:06.233148 3431 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260809 15:40:06.238448 26872 catalog_manager.cc:1347] Generated new cluster ID: 13d095f417b4434f8c675c45edb475db
I20260809 15:40:06.238507 26872 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 15:40:06.243440 26856 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:40:06.235333 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36737 (local address 127.3.89.254:42641)
0809 15:40:06.235430 (+ 97us) server_negotiation.cc:206] Beginning negotiation
0809 15:40:06.235434 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 15:40:06.235644 (+ 210us) server_negotiation.cc:407] Connection header received
0809 15:40:06.235738 (+ 94us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 15:40:06.235741 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 15:40:06.235784 (+ 43us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 15:40:06.235869 (+ 85us) server_negotiation.cc:226] Negotiated authn=SASL
0809 15:40:06.236599 (+ 730us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:40:06.237355 (+ 756us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:40:06.238444 (+ 1089us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:40:06.238769 (+ 325us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:40:06.238878 (+ 109us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:40:06.240220 (+ 1342us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 15:40:06.240224 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 15:40:06.240226 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 15:40:06.240249 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0809 15:40:06.241635 (+ 1386us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 15:40:06.242100 (+ 465us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 15:40:06.242104 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 15:40:06.242106 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 15:40:06.242183 (+ 77us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 15:40:06.242507 (+ 324us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 15:40:06.242511 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 15:40:06.242512 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 15:40:06.242668 (+ 156us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 15:40:06.242819 (+ 151us) server_negotiation.cc:1035] Waiting for connection context
0809 15:40:06.243153 (+ 334us) server_negotiation.cc:299] Negotiation successful
0809 15:40:06.243244 (+ 91us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":40}
I20260809 15:40:06.248467 26872 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 15:40:06.249259 26872 catalog_manager.cc:1504] Loading token signing keys...
I20260809 15:40:06.260912 26872 catalog_manager.cc:5868] T 00000000000000000000000000000000 P c5f7169ee54c4588ab91659c5ada193d: Generated new TSK 0
I20260809 15:40:06.261591 26872 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260809 15:40:06.267483 3431 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskm314YR/build/debug/bin/kudu with pid 26778
W20260809 15:40:06.274930 26761 connection.cc:535] client connection to 127.3.89.254:42641 recv error: Network error: failed to read from TLS socket (remote: 127.3.89.254:42641): Connection reset by peer (error 104)
2026-08-09T15:40:06Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/12 (1480 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/13
Loading random data
Initializing database '/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786289732823074-3431-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 15:40:06 dist-test-slave-2w3w krb5kdc[26889](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 15:40:06 dist-test-slave-2w3w krb5kdc[26889](info): set up 2 sockets
Aug 09 15:40:06 dist-test-slave-2w3w krb5kdc[26889](info): commencing operation
krb5kdc: starting...
W20260809 15:40:07.344655 3431 mini_kdc.cc:121] Time spent starting KDC: real 1.055s user 0.000s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786289732823074-3431-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786289732823074-3431-0/krb5kdc/test-user.keytab.
Aug 09 15:40:07 dist-test-slave-2w3w krb5kdc[26889](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786290007, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T15:40:07Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T15:40:07Z Disabled control of system clock
WARNING: no policy specified for kudu/127.3.89.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.3.89.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.3.89.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786289732823074-3431-0/krb5kdc/kudu_127.3.89.254.keytab.
Entry for principal kudu/127.3.89.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786289732823074-3431-0/krb5kdc/kudu_127.3.89.254.keytab.
I20260809 15:40:07.467782 3431 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskm314YR/build/debug/bin/kudu
/tmp/dist-test-taskm314YR/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786289732823074-3431-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786289732823074-3431-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786289732823074-3431-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786289732823074-3431-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.3.89.254:41685
--webserver_interface=127.3.89.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786289732823074-3431-0/krb5kdc/kudu_127.3.89.254.keytab
--principal=kudu/127.3.89.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.3.89.212:34121
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.3.89.254:41685
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786289732823074-3431-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786289732823074-3431-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786289732823074-3431-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 15:40:07.578894 26902 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 15:40:07.579229 26902 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 15:40:07.579308 26902 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 15:40:07.583292 26902 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 15:40:07.583364 26902 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 15:40:07.583400 26902 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 15:40:07.583451 26902 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 15:40:07.583489 26902 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 15:40:07.588655 26902 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.3.89.212:34121
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786289732823074-3431-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786289732823074-3431-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.3.89.254:41685
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786289732823074-3431-0/krb5kdc/kudu_127.3.89.254.keytab
--principal=kudu/127.3.89.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.3.89.254:41685
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786289732823074-3431-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.3.89.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.26902
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786289732823074-3431-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 09 Aug 2026 14:46:55 UTC on ccycloud.quasar-yswkmf.root.comops.site
build id 4576540
I20260809 15:40:07.589668 26902 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 15:40:07.590695 26902 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 15:40:07.599242 26909 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 15:40:07.599254 26911 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 15:40:07.599252 26908 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 15:40:07.599524 26902 server_base.cc:1034] running on GCE node
I20260809 15:40:07.600032 26902 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 15:40:07.601044 26902 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 15:40:07.602205 26902 hybrid_clock.cc:648] HybridClock initialized: now 1786290007602191 us; error 25 us; skew 500 ppm
Aug 09 15:40:07 dist-test-slave-2w3w krb5kdc[26889](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786290007, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.89.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 15:40:07.610352 26902 init.cc:377] Logged in from keytab as kudu/127.3.89.254@KRBTEST.COM (short username kudu)
I20260809 15:40:07.611277 26902 webserver.cc:460] Webserver started at http://127.3.89.254:33773/ using document root <none> and password file <none>
I20260809 15:40:07.611822 26902 fs_manager.cc:359] Metadata directory not provided
I20260809 15:40:07.611912 26902 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 15:40:07.612164 26902 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 15:40:07.614192 26902 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskm314YR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786289732823074-3431-0/minicluster-data/master-0/data/instance:
uuid: "e00948da08014df6b11ceb752a4a4560"
format_stamp: "Formatted at 2026-08-09 15:40:07 on dist-test-slave-2w3w"
server_key: "5d752d5d207104f92844af6d016bdcf7"
server_key_iv: "ad84f85f785ae51bb6c1ef3361a59dc7"
server_key_version: "encryptionkey@0"
I20260809 15:40:07.614777 26902 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskm314YR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786289732823074-3431-0/minicluster-data/master-0/wal/instance:
uuid: "e00948da08014df6b11ceb752a4a4560"
format_stamp: "Formatted at 2026-08-09 15:40:07 on dist-test-slave-2w3w"
server_key: "5d752d5d207104f92844af6d016bdcf7"
server_key_iv: "ad84f85f785ae51bb6c1ef3361a59dc7"
server_key_version: "encryptionkey@0"
I20260809 15:40:07.618309 26902 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.002s sys 0.000s
I20260809 15:40:07.620573 26921 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 15:40:07.621726 26902 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260809 15:40:07.621868 26902 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskm314YR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786289732823074-3431-0/minicluster-data/master-0/data,/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786289732823074-3431-0/minicluster-data/master-0/wal
uuid: "e00948da08014df6b11ceb752a4a4560"
format_stamp: "Formatted at 2026-08-09 15:40:07 on dist-test-slave-2w3w"
server_key: "5d752d5d207104f92844af6d016bdcf7"
server_key_iv: "ad84f85f785ae51bb6c1ef3361a59dc7"
server_key_version: "encryptionkey@0"
I20260809 15:40:07.621982 26902 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskm314YR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786289732823074-3431-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskm314YR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786289732823074-3431-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskm314YR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786289732823074-3431-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 15:40:07.636111 26902 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 15:40:07.638670 26902 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 15:40:07.638877 26902 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 15:40:07.646785 26902 rpc_server.cc:307] RPC server started. Bound to: 127.3.89.254:41685
I20260809 15:40:07.646821 26980 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.3.89.254:41685 every 8 connection(s)
I20260809 15:40:07.647744 26902 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskm314YR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786289732823074-3431-0/minicluster-data/master-0/data/info.pb
I20260809 15:40:07.650337 26982 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 15:40:07.653689 3431 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskm314YR/build/debug/bin/kudu as pid 26902
I20260809 15:40:07.653796 3431 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskm314YR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786289732823074-3431-0/minicluster-data/master-0/wal/instance
I20260809 15:40:07.654052 3431 external_mini_cluster.cc:1428] Setting key 775f07770a5b2ed3026e85472b41f6dd
I20260809 15:40:07.656163 26982 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e00948da08014df6b11ceb752a4a4560: Bootstrap starting.
I20260809 15:40:07.658736 26982 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P e00948da08014df6b11ceb752a4a4560: Neither blocks nor log segments found. Creating new log.
I20260809 15:40:07.659597 26982 log.cc:824] T 00000000000000000000000000000000 P e00948da08014df6b11ceb752a4a4560: Log is configured to *not* fsync() on all Append() calls
I20260809 15:40:07.661612 26982 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e00948da08014df6b11ceb752a4a4560: No bootstrap required, opened a new log
I20260809 15:40:07.664216 26985 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:40:07.655419 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42191 (local address 127.3.89.254:41685)
0809 15:40:07.655829 (+ 410us) server_negotiation.cc:206] Beginning negotiation
0809 15:40:07.655847 (+ 18us) server_negotiation.cc:399] Waiting for connection header
0809 15:40:07.655887 (+ 40us) server_negotiation.cc:407] Connection header received
0809 15:40:07.656735 (+ 848us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 15:40:07.656757 (+ 22us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 15:40:07.657162 (+ 405us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 15:40:07.657539 (+ 377us) server_negotiation.cc:226] Negotiated authn=SASL
0809 15:40:07.657937 (+ 398us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:40:07.659396 (+ 1459us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:40:07.660456 (+ 1060us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:40:07.661096 (+ 640us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:40:07.661249 (+ 153us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:40:07.662021 (+ 772us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 15:40:07.662027 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 15:40:07.662040 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 15:40:07.662521 (+ 481us) server_negotiation.cc:922] calling sasl_server_start()
0809 15:40:07.662593 (+ 72us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 15:40:07.662622 (+ 29us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 15:40:07.662759 (+ 137us) server_negotiation.cc:1035] Waiting for connection context
0809 15:40:07.662939 (+ 180us) server_negotiation.cc:299] Negotiation successful
0809 15:40:07.663091 (+ 152us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":264,"thread_start_us":109,"threads_started":1}
I20260809 15:40:07.665529 26982 raft_consensus.cc:348] T 00000000000000000000000000000000 P e00948da08014df6b11ceb752a4a4560 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e00948da08014df6b11ceb752a4a4560" member_type: VOTER last_known_addr { host: "127.3.89.254" port: 41685 } }
I20260809 15:40:07.665723 26982 raft_consensus.cc:374] T 00000000000000000000000000000000 P e00948da08014df6b11ceb752a4a4560 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 15:40:07.665784 26982 raft_consensus.cc:729] T 00000000000000000000000000000000 P e00948da08014df6b11ceb752a4a4560 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e00948da08014df6b11ceb752a4a4560, State: Initialized, Role: FOLLOWER
I20260809 15:40:07.666253 26982 consensus_queue.cc:260] T 00000000000000000000000000000000 P e00948da08014df6b11ceb752a4a4560 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e00948da08014df6b11ceb752a4a4560" member_type: VOTER last_known_addr { host: "127.3.89.254" port: 41685 } }
I20260809 15:40:07.666409 26982 raft_consensus.cc:388] T 00000000000000000000000000000000 P e00948da08014df6b11ceb752a4a4560 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 15:40:07.666483 26982 raft_consensus.cc:482] T 00000000000000000000000000000000 P e00948da08014df6b11ceb752a4a4560 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 15:40:07.666581 26982 raft_consensus.cc:3037] T 00000000000000000000000000000000 P e00948da08014df6b11ceb752a4a4560 [term 0 FOLLOWER]: Advancing to term 1
I20260809 15:40:07.667125 3431 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260809 15:40:07.667562 26982 raft_consensus.cc:504] T 00000000000000000000000000000000 P e00948da08014df6b11ceb752a4a4560 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e00948da08014df6b11ceb752a4a4560" member_type: VOTER last_known_addr { host: "127.3.89.254" port: 41685 } }
I20260809 15:40:07.667903 26982 leader_election.cc:302] T 00000000000000000000000000000000 P e00948da08014df6b11ceb752a4a4560 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: e00948da08014df6b11ceb752a4a4560; no voters:
I20260809 15:40:07.668208 26982 leader_election.cc:288] T 00000000000000000000000000000000 P e00948da08014df6b11ceb752a4a4560 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 15:40:07.668315 26987 raft_consensus.cc:2781] T 00000000000000000000000000000000 P e00948da08014df6b11ceb752a4a4560 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 15:40:07.668637 26987 raft_consensus.cc:686] T 00000000000000000000000000000000 P e00948da08014df6b11ceb752a4a4560 [term 1 LEADER]: Becoming Leader. State: Replica: e00948da08014df6b11ceb752a4a4560, State: Running, Role: LEADER
I20260809 15:40:07.668946 26987 consensus_queue.cc:237] T 00000000000000000000000000000000 P e00948da08014df6b11ceb752a4a4560 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e00948da08014df6b11ceb752a4a4560" member_type: VOTER last_known_addr { host: "127.3.89.254" port: 41685 } }
I20260809 15:40:07.669399 26982 sys_catalog.cc:564] T 00000000000000000000000000000000 P e00948da08014df6b11ceb752a4a4560 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 15:40:07.670181 26987 sys_catalog.cc:455] T 00000000000000000000000000000000 P e00948da08014df6b11ceb752a4a4560 [sys.catalog]: SysCatalogTable state changed. Reason: New leader e00948da08014df6b11ceb752a4a4560. Latest consensus state: current_term: 1 leader_uuid: "e00948da08014df6b11ceb752a4a4560" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e00948da08014df6b11ceb752a4a4560" member_type: VOTER last_known_addr { host: "127.3.89.254" port: 41685 } } }
I20260809 15:40:07.670367 26987 sys_catalog.cc:458] T 00000000000000000000000000000000 P e00948da08014df6b11ceb752a4a4560 [sys.catalog]: This master's current role is: LEADER
I20260809 15:40:07.670599 26991 sys_catalog.cc:455] T 00000000000000000000000000000000 P e00948da08014df6b11ceb752a4a4560 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "e00948da08014df6b11ceb752a4a4560" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e00948da08014df6b11ceb752a4a4560" member_type: VOTER last_known_addr { host: "127.3.89.254" port: 41685 } } }
I20260809 15:40:07.670760 26991 sys_catalog.cc:458] T 00000000000000000000000000000000 P e00948da08014df6b11ceb752a4a4560 [sys.catalog]: This master's current role is: LEADER
I20260809 15:40:07.670782 27001 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 15:40:07.674208 26985 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:40:07.669364 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58801 (local address 127.3.89.254:41685)
0809 15:40:07.669485 (+ 121us) server_negotiation.cc:206] Beginning negotiation
0809 15:40:07.669489 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 15:40:07.669778 (+ 289us) server_negotiation.cc:407] Connection header received
0809 15:40:07.669869 (+ 91us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 15:40:07.669874 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 15:40:07.669921 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 15:40:07.670004 (+ 83us) server_negotiation.cc:226] Negotiated authn=SASL
0809 15:40:07.670320 (+ 316us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:40:07.671115 (+ 795us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:40:07.672244 (+ 1129us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:40:07.672573 (+ 329us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:40:07.672647 (+ 74us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:40:07.673706 (+ 1059us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 15:40:07.673710 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 15:40:07.673713 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 15:40:07.673725 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0809 15:40:07.673751 (+ 26us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 15:40:07.673766 (+ 15us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 15:40:07.673857 (+ 91us) server_negotiation.cc:1035] Waiting for connection context
0809 15:40:07.674051 (+ 194us) server_negotiation.cc:299] Negotiation successful
0809 15:40:07.674086 (+ 35us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":37}
I20260809 15:40:07.674393 27001 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 15:40:07.679490 27001 catalog_manager.cc:1347] Generated new cluster ID: d9bb83a0bb5a4852a9c9cffd4a7c62f2
I20260809 15:40:07.679607 27001 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 15:40:07.702464 27001 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 15:40:07.703471 27001 catalog_manager.cc:1504] Loading token signing keys...
I20260809 15:40:07.709038 27001 catalog_manager.cc:5868] T 00000000000000000000000000000000 P e00948da08014df6b11ceb752a4a4560: Generated new TSK 0
I20260809 15:40:07.709662 27001 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260809 15:40:07.718196 3431 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskm314YR/build/debug/bin/kudu with pid 26902
W20260809 15:40:07.725169 26886 connection.cc:535] client connection to 127.3.89.254:41685 recv error: Network error: failed to read from TLS socket (remote: 127.3.89.254:41685): Connection reset by peer (error 104)
2026-08-09T15:40:07Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/13 (1450 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/14
Loading random data
Initializing database '/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786289732823074-3431-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 15:40:07 dist-test-slave-2w3w krb5kdc[27016](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 15:40:07 dist-test-slave-2w3w krb5kdc[27016](info): set up 2 sockets
Aug 09 15:40:07 dist-test-slave-2w3w krb5kdc[27016](info): commencing operation
krb5kdc: starting...
W20260809 15:40:08.850085 3431 mini_kdc.cc:121] Time spent starting KDC: real 1.111s user 0.000s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786289732823074-3431-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786289732823074-3431-0/krb5kdc/test-user.keytab.
Aug 09 15:40:08 dist-test-slave-2w3w krb5kdc[27016](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786290008, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T15:40:08Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T15:40:08Z Disabled control of system clock
WARNING: no policy specified for kudu/127.3.89.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.3.89.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.3.89.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786289732823074-3431-0/krb5kdc/kudu_127.3.89.254.keytab.
Entry for principal kudu/127.3.89.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786289732823074-3431-0/krb5kdc/kudu_127.3.89.254.keytab.
I20260809 15:40:08.974463 3431 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskm314YR/build/debug/bin/kudu
/tmp/dist-test-taskm314YR/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786289732823074-3431-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786289732823074-3431-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786289732823074-3431-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786289732823074-3431-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.3.89.254:37749
--webserver_interface=127.3.89.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786289732823074-3431-0/krb5kdc/kudu_127.3.89.254.keytab
--principal=kudu/127.3.89.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.3.89.212:34487
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.3.89.254:37749
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786289732823074-3431-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786289732823074-3431-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786289732823074-3431-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 15:40:09.092232 27032 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 15:40:09.092521 27032 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 15:40:09.092592 27032 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 15:40:09.096622 27032 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 15:40:09.096702 27032 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 15:40:09.096748 27032 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 15:40:09.096799 27032 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 15:40:09.096841 27032 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 15:40:09.102030 27032 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.3.89.212:34487
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786289732823074-3431-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786289732823074-3431-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.3.89.254:37749
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786289732823074-3431-0/krb5kdc/kudu_127.3.89.254.keytab
--principal=kudu/127.3.89.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.3.89.254:37749
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786289732823074-3431-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.3.89.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.27032
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786289732823074-3431-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 09 Aug 2026 14:46:55 UTC on ccycloud.quasar-yswkmf.root.comops.site
build id 4576540
I20260809 15:40:09.103147 27032 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 15:40:09.104162 27032 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 15:40:09.112557 27040 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 15:40:09.112598 27042 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 15:40:09.112613 27039 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 15:40:09.112928 27032 server_base.cc:1034] running on GCE node
I20260809 15:40:09.113502 27032 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 15:40:09.114581 27032 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 15:40:09.115764 27032 hybrid_clock.cc:648] HybridClock initialized: now 1786290009115764 us; error 42 us; skew 500 ppm
Aug 09 15:40:09 dist-test-slave-2w3w krb5kdc[27016](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786290009, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.89.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 15:40:09.118436 27032 init.cc:377] Logged in from keytab as kudu/127.3.89.254@KRBTEST.COM (short username kudu)
I20260809 15:40:09.119387 27032 webserver.cc:460] Webserver started at http://127.3.89.254:35417/ using document root <none> and password file <none>
I20260809 15:40:09.119935 27032 fs_manager.cc:359] Metadata directory not provided
I20260809 15:40:09.120036 27032 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 15:40:09.120265 27032 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 15:40:09.122264 27032 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskm314YR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786289732823074-3431-0/minicluster-data/master-0/data/instance:
uuid: "c17f44fc1f5f435593178e960b495ef8"
format_stamp: "Formatted at 2026-08-09 15:40:09 on dist-test-slave-2w3w"
server_key: "1c1c2592aaa7528b9bc277ea8a31a2f7"
server_key_iv: "91ee7c2bb1e13ba527685015563cd4d4"
server_key_version: "encryptionkey@0"
I20260809 15:40:09.122841 27032 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskm314YR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786289732823074-3431-0/minicluster-data/master-0/wal/instance:
uuid: "c17f44fc1f5f435593178e960b495ef8"
format_stamp: "Formatted at 2026-08-09 15:40:09 on dist-test-slave-2w3w"
server_key: "1c1c2592aaa7528b9bc277ea8a31a2f7"
server_key_iv: "91ee7c2bb1e13ba527685015563cd4d4"
server_key_version: "encryptionkey@0"
I20260809 15:40:09.126436 27032 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.002s sys 0.000s
I20260809 15:40:09.128751 27049 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 15:40:09.129949 27032 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260809 15:40:09.130088 27032 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskm314YR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786289732823074-3431-0/minicluster-data/master-0/data,/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786289732823074-3431-0/minicluster-data/master-0/wal
uuid: "c17f44fc1f5f435593178e960b495ef8"
format_stamp: "Formatted at 2026-08-09 15:40:09 on dist-test-slave-2w3w"
server_key: "1c1c2592aaa7528b9bc277ea8a31a2f7"
server_key_iv: "91ee7c2bb1e13ba527685015563cd4d4"
server_key_version: "encryptionkey@0"
I20260809 15:40:09.130192 27032 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskm314YR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786289732823074-3431-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskm314YR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786289732823074-3431-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskm314YR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786289732823074-3431-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 15:40:09.134665 27032 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 15:40:09.137458 27032 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 15:40:09.137652 27032 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 15:40:09.144799 27032 rpc_server.cc:307] RPC server started. Bound to: 127.3.89.254:37749
I20260809 15:40:09.144827 27107 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.3.89.254:37749 every 8 connection(s)
I20260809 15:40:09.145679 27032 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskm314YR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786289732823074-3431-0/minicluster-data/master-0/data/info.pb
I20260809 15:40:09.148345 27108 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 15:40:09.151206 3431 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskm314YR/build/debug/bin/kudu as pid 27032
I20260809 15:40:09.151381 3431 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskm314YR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786289732823074-3431-0/minicluster-data/master-0/wal/instance
I20260809 15:40:09.151675 3431 external_mini_cluster.cc:1428] Setting key 36360fb8808d78a1b1e85dc0a01b88dd
I20260809 15:40:09.154232 27108 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c17f44fc1f5f435593178e960b495ef8: Bootstrap starting.
I20260809 15:40:09.156556 27108 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P c17f44fc1f5f435593178e960b495ef8: Neither blocks nor log segments found. Creating new log.
I20260809 15:40:09.156636 27112 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:40:09.153018 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43769 (local address 127.3.89.254:37749)
0809 15:40:09.153427 (+ 409us) server_negotiation.cc:206] Beginning negotiation
0809 15:40:09.153440 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0809 15:40:09.153483 (+ 43us) server_negotiation.cc:407] Connection header received
0809 15:40:09.154069 (+ 586us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 15:40:09.154084 (+ 15us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 15:40:09.154388 (+ 304us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 15:40:09.154707 (+ 319us) server_negotiation.cc:226] Negotiated authn=SASL
0809 15:40:09.155494 (+ 787us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 15:40:09.155504 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 15:40:09.155515 (+ 11us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 15:40:09.155527 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0809 15:40:09.155580 (+ 53us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 15:40:09.155599 (+ 19us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 15:40:09.155744 (+ 145us) server_negotiation.cc:1035] Waiting for connection context
0809 15:40:09.155812 (+ 68us) server_negotiation.cc:299] Negotiation successful
0809 15:40:09.155956 (+ 144us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":273,"thread_start_us":122,"threads_started":1}
I20260809 15:40:09.157437 27108 log.cc:824] T 00000000000000000000000000000000 P c17f44fc1f5f435593178e960b495ef8: Log is configured to *not* fsync() on all Append() calls
I20260809 15:40:09.159371 3431 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260809 15:40:09.159550 27108 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c17f44fc1f5f435593178e960b495ef8: No bootstrap required, opened a new log
I20260809 15:40:09.163022 27112 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:40:09.161423 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53699 (local address 127.3.89.254:37749)
0809 15:40:09.161546 (+ 123us) server_negotiation.cc:206] Beginning negotiation
0809 15:40:09.161550 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 15:40:09.161726 (+ 176us) server_negotiation.cc:407] Connection header received
0809 15:40:09.161838 (+ 112us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 15:40:09.161840 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 15:40:09.161865 (+ 25us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 15:40:09.161927 (+ 62us) server_negotiation.cc:226] Negotiated authn=SASL
0809 15:40:09.162632 (+ 705us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 15:40:09.162634 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 15:40:09.162636 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 15:40:09.162646 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0809 15:40:09.162669 (+ 23us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 15:40:09.162678 (+ 9us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 15:40:09.162745 (+ 67us) server_negotiation.cc:1035] Waiting for connection context
0809 15:40:09.162838 (+ 93us) server_negotiation.cc:299] Negotiation successful
0809 15:40:09.162867 (+ 29us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":48}
I20260809 15:40:09.163663 27108 raft_consensus.cc:348] T 00000000000000000000000000000000 P c17f44fc1f5f435593178e960b495ef8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c17f44fc1f5f435593178e960b495ef8" member_type: VOTER last_known_addr { host: "127.3.89.254" port: 37749 } }
I20260809 15:40:09.163908 27108 raft_consensus.cc:374] T 00000000000000000000000000000000 P c17f44fc1f5f435593178e960b495ef8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 15:40:09.163964 27108 raft_consensus.cc:729] T 00000000000000000000000000000000 P c17f44fc1f5f435593178e960b495ef8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c17f44fc1f5f435593178e960b495ef8, State: Initialized, Role: FOLLOWER
I20260809 15:40:09.164341 27108 consensus_queue.cc:260] T 00000000000000000000000000000000 P c17f44fc1f5f435593178e960b495ef8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c17f44fc1f5f435593178e960b495ef8" member_type: VOTER last_known_addr { host: "127.3.89.254" port: 37749 } }
I20260809 15:40:09.164482 27108 raft_consensus.cc:388] T 00000000000000000000000000000000 P c17f44fc1f5f435593178e960b495ef8 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 15:40:09.164547 27108 raft_consensus.cc:482] T 00000000000000000000000000000000 P c17f44fc1f5f435593178e960b495ef8 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 15:40:09.164639 27108 raft_consensus.cc:3037] T 00000000000000000000000000000000 P c17f44fc1f5f435593178e960b495ef8 [term 0 FOLLOWER]: Advancing to term 1
I20260809 15:40:09.165484 27108 raft_consensus.cc:504] T 00000000000000000000000000000000 P c17f44fc1f5f435593178e960b495ef8 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c17f44fc1f5f435593178e960b495ef8" member_type: VOTER last_known_addr { host: "127.3.89.254" port: 37749 } }
I20260809 15:40:09.165813 27108 leader_election.cc:302] T 00000000000000000000000000000000 P c17f44fc1f5f435593178e960b495ef8 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: c17f44fc1f5f435593178e960b495ef8; no voters:
I20260809 15:40:09.166093 27108 leader_election.cc:288] T 00000000000000000000000000000000 P c17f44fc1f5f435593178e960b495ef8 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 15:40:09.166191 27120 raft_consensus.cc:2781] T 00000000000000000000000000000000 P c17f44fc1f5f435593178e960b495ef8 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 15:40:09.166441 27120 raft_consensus.cc:686] T 00000000000000000000000000000000 P c17f44fc1f5f435593178e960b495ef8 [term 1 LEADER]: Becoming Leader. State: Replica: c17f44fc1f5f435593178e960b495ef8, State: Running, Role: LEADER
I20260809 15:40:09.166790 27120 consensus_queue.cc:237] T 00000000000000000000000000000000 P c17f44fc1f5f435593178e960b495ef8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c17f44fc1f5f435593178e960b495ef8" member_type: VOTER last_known_addr { host: "127.3.89.254" port: 37749 } }
I20260809 15:40:09.167321 27108 sys_catalog.cc:564] T 00000000000000000000000000000000 P c17f44fc1f5f435593178e960b495ef8 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 15:40:09.168362 27122 sys_catalog.cc:455] T 00000000000000000000000000000000 P c17f44fc1f5f435593178e960b495ef8 [sys.catalog]: SysCatalogTable state changed. Reason: New leader c17f44fc1f5f435593178e960b495ef8. Latest consensus state: current_term: 1 leader_uuid: "c17f44fc1f5f435593178e960b495ef8" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c17f44fc1f5f435593178e960b495ef8" member_type: VOTER last_known_addr { host: "127.3.89.254" port: 37749 } } }
I20260809 15:40:09.168498 27122 sys_catalog.cc:458] T 00000000000000000000000000000000 P c17f44fc1f5f435593178e960b495ef8 [sys.catalog]: This master's current role is: LEADER
I20260809 15:40:09.168876 27130 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 15:40:09.168833 27121 sys_catalog.cc:455] T 00000000000000000000000000000000 P c17f44fc1f5f435593178e960b495ef8 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "c17f44fc1f5f435593178e960b495ef8" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c17f44fc1f5f435593178e960b495ef8" member_type: VOTER last_known_addr { host: "127.3.89.254" port: 37749 } } }
I20260809 15:40:09.168983 27121 sys_catalog.cc:458] T 00000000000000000000000000000000 P c17f44fc1f5f435593178e960b495ef8 [sys.catalog]: This master's current role is: LEADER
I20260809 15:40:09.171944 27130 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 15:40:09.176906 27130 catalog_manager.cc:1347] Generated new cluster ID: 6bd297df341d40a1aedaf00af204e114
I20260809 15:40:09.176970 27130 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 15:40:09.192955 27130 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 15:40:09.193274 27130 catalog_manager.cc:1504] Loading token signing keys...
I20260809 15:40:09.200155 27130 catalog_manager.cc:5868] T 00000000000000000000000000000000 P c17f44fc1f5f435593178e960b495ef8: Generated new TSK 0
I20260809 15:40:09.200891 27130 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260809 15:40:09.207481 3431 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskm314YR/build/debug/bin/kudu with pid 27032
2026-08-09T15:40:09Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/14 (1491 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/15
Loading random data
Initializing database '/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786289732823074-3431-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 15:40:09 dist-test-slave-2w3w krb5kdc[27142](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 15:40:09 dist-test-slave-2w3w krb5kdc[27142](info): set up 2 sockets
Aug 09 15:40:09 dist-test-slave-2w3w krb5kdc[27142](info): commencing operation
krb5kdc: starting...
W20260809 15:40:10.289053 3431 mini_kdc.cc:121] Time spent starting KDC: real 1.058s user 0.000s sys 0.008s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786289732823074-3431-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786289732823074-3431-0/krb5kdc/test-user.keytab.
Aug 09 15:40:10 dist-test-slave-2w3w krb5kdc[27142](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786290010, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T15:40:10Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T15:40:10Z Disabled control of system clock
WARNING: no policy specified for kudu/127.3.89.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.3.89.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.3.89.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786289732823074-3431-0/krb5kdc/kudu_127.3.89.254.keytab.
Entry for principal kudu/127.3.89.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786289732823074-3431-0/krb5kdc/kudu_127.3.89.254.keytab.
I20260809 15:40:10.412000 3431 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskm314YR/build/debug/bin/kudu
/tmp/dist-test-taskm314YR/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786289732823074-3431-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786289732823074-3431-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786289732823074-3431-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786289732823074-3431-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.3.89.254:43123
--webserver_interface=127.3.89.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786289732823074-3431-0/krb5kdc/kudu_127.3.89.254.keytab
--principal=kudu/127.3.89.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.3.89.212:37119
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.3.89.254:43123
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786289732823074-3431-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786289732823074-3431-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786289732823074-3431-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 15:40:10.522594 27156 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 15:40:10.522853 27156 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 15:40:10.522893 27156 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 15:40:10.526844 27156 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 15:40:10.526906 27156 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 15:40:10.526926 27156 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 15:40:10.526943 27156 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 15:40:10.526974 27156 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 15:40:10.532369 27156 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.3.89.212:37119
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786289732823074-3431-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786289732823074-3431-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.3.89.254:43123
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786289732823074-3431-0/krb5kdc/kudu_127.3.89.254.keytab
--principal=kudu/127.3.89.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.3.89.254:43123
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786289732823074-3431-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.3.89.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.27156
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786289732823074-3431-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 09 Aug 2026 14:46:55 UTC on ccycloud.quasar-yswkmf.root.comops.site
build id 4576540
I20260809 15:40:10.533329 27156 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 15:40:10.534621 27156 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 15:40:10.542922 27165 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 15:40:10.542932 27163 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 15:40:10.542927 27161 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 15:40:10.543140 27156 server_base.cc:1034] running on GCE node
I20260809 15:40:10.543792 27156 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 15:40:10.544828 27156 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 15:40:10.545986 27156 hybrid_clock.cc:648] HybridClock initialized: now 1786290010545967 us; error 31 us; skew 500 ppm
Aug 09 15:40:10 dist-test-slave-2w3w krb5kdc[27142](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786290010, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.89.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 15:40:10.548710 27156 init.cc:377] Logged in from keytab as kudu/127.3.89.254@KRBTEST.COM (short username kudu)
I20260809 15:40:10.549504 27156 webserver.cc:460] Webserver started at http://127.3.89.254:44729/ using document root <none> and password file <none>
I20260809 15:40:10.549989 27156 fs_manager.cc:359] Metadata directory not provided
I20260809 15:40:10.550047 27156 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 15:40:10.550203 27156 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 15:40:10.552145 27156 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskm314YR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786289732823074-3431-0/minicluster-data/master-0/data/instance:
uuid: "a7daa656c4dd455a8a2880b8230a716f"
format_stamp: "Formatted at 2026-08-09 15:40:10 on dist-test-slave-2w3w"
server_key: "afd173f9e77604de3e0bef491e025e33"
server_key_iv: "724870e7ab4b0f2ab90069d8733a0601"
server_key_version: "encryptionkey@0"
I20260809 15:40:10.552652 27156 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskm314YR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786289732823074-3431-0/minicluster-data/master-0/wal/instance:
uuid: "a7daa656c4dd455a8a2880b8230a716f"
format_stamp: "Formatted at 2026-08-09 15:40:10 on dist-test-slave-2w3w"
server_key: "afd173f9e77604de3e0bef491e025e33"
server_key_iv: "724870e7ab4b0f2ab90069d8733a0601"
server_key_version: "encryptionkey@0"
I20260809 15:40:10.556035 27156 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.004s sys 0.000s
I20260809 15:40:10.558224 27174 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 15:40:10.559407 27156 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260809 15:40:10.559548 27156 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskm314YR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786289732823074-3431-0/minicluster-data/master-0/data,/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786289732823074-3431-0/minicluster-data/master-0/wal
uuid: "a7daa656c4dd455a8a2880b8230a716f"
format_stamp: "Formatted at 2026-08-09 15:40:10 on dist-test-slave-2w3w"
server_key: "afd173f9e77604de3e0bef491e025e33"
server_key_iv: "724870e7ab4b0f2ab90069d8733a0601"
server_key_version: "encryptionkey@0"
I20260809 15:40:10.559652 27156 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskm314YR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786289732823074-3431-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskm314YR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786289732823074-3431-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskm314YR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786289732823074-3431-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 15:40:10.574571 27156 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 15:40:10.577483 27156 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 15:40:10.577670 27156 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 15:40:10.584502 27156 rpc_server.cc:307] RPC server started. Bound to: 127.3.89.254:43123
I20260809 15:40:10.584581 27230 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.3.89.254:43123 every 8 connection(s)
I20260809 15:40:10.585407 27156 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskm314YR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786289732823074-3431-0/minicluster-data/master-0/data/info.pb
I20260809 15:40:10.588061 3431 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskm314YR/build/debug/bin/kudu as pid 27156
I20260809 15:40:10.588203 3431 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskm314YR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786289732823074-3431-0/minicluster-data/master-0/wal/instance
I20260809 15:40:10.588330 27231 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 15:40:10.588521 3431 external_mini_cluster.cc:1428] Setting key 85fb59d3cd5c2ef41421c56334287419
I20260809 15:40:10.595575 27231 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a7daa656c4dd455a8a2880b8230a716f: Bootstrap starting.
Aug 09 15:40:10 dist-test-slave-2w3w krb5kdc[27142](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786290010, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.3.89.254@KRBTEST.COM
I20260809 15:40:10.597862 27231 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P a7daa656c4dd455a8a2880b8230a716f: Neither blocks nor log segments found. Creating new log.
I20260809 15:40:10.598680 27231 log.cc:824] T 00000000000000000000000000000000 P a7daa656c4dd455a8a2880b8230a716f: Log is configured to *not* fsync() on all Append() calls
I20260809 15:40:10.600510 27231 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a7daa656c4dd455a8a2880b8230a716f: No bootstrap required, opened a new log
I20260809 15:40:10.602934 27234 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:40:10.589921 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55447 (local address 127.3.89.254:43123)
0809 15:40:10.590367 (+ 446us) server_negotiation.cc:206] Beginning negotiation
0809 15:40:10.590386 (+ 19us) server_negotiation.cc:399] Waiting for connection header
0809 15:40:10.590428 (+ 42us) server_negotiation.cc:407] Connection header received
0809 15:40:10.591174 (+ 746us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 15:40:10.591198 (+ 24us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 15:40:10.591620 (+ 422us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 15:40:10.591995 (+ 375us) server_negotiation.cc:226] Negotiated authn=SASL
0809 15:40:10.592690 (+ 695us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:40:10.594187 (+ 1497us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:40:10.595082 (+ 895us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:40:10.595775 (+ 693us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:40:10.595941 (+ 166us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:40:10.598390 (+ 2449us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 15:40:10.598396 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 15:40:10.598410 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 15:40:10.598440 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0809 15:40:10.600225 (+ 1785us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 15:40:10.600687 (+ 462us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 15:40:10.600692 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 15:40:10.600696 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 15:40:10.600809 (+ 113us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 15:40:10.601092 (+ 283us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 15:40:10.601096 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 15:40:10.601097 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 15:40:10.601474 (+ 377us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 15:40:10.601673 (+ 199us) server_negotiation.cc:1035] Waiting for connection context
0809 15:40:10.601963 (+ 290us) server_negotiation.cc:299] Negotiation successful
0809 15:40:10.602244 (+ 281us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":305,"thread_start_us":133,"threads_started":1}
I20260809 15:40:10.604208 27231 raft_consensus.cc:348] T 00000000000000000000000000000000 P a7daa656c4dd455a8a2880b8230a716f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a7daa656c4dd455a8a2880b8230a716f" member_type: VOTER last_known_addr { host: "127.3.89.254" port: 43123 } }
I20260809 15:40:10.604470 27231 raft_consensus.cc:374] T 00000000000000000000000000000000 P a7daa656c4dd455a8a2880b8230a716f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 15:40:10.604519 27231 raft_consensus.cc:729] T 00000000000000000000000000000000 P a7daa656c4dd455a8a2880b8230a716f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a7daa656c4dd455a8a2880b8230a716f, State: Initialized, Role: FOLLOWER
I20260809 15:40:10.604985 27231 consensus_queue.cc:260] T 00000000000000000000000000000000 P a7daa656c4dd455a8a2880b8230a716f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a7daa656c4dd455a8a2880b8230a716f" member_type: VOTER last_known_addr { host: "127.3.89.254" port: 43123 } }
I20260809 15:40:10.605160 27231 raft_consensus.cc:388] T 00000000000000000000000000000000 P a7daa656c4dd455a8a2880b8230a716f [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 15:40:10.605240 27231 raft_consensus.cc:482] T 00000000000000000000000000000000 P a7daa656c4dd455a8a2880b8230a716f [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 15:40:10.605345 27231 raft_consensus.cc:3037] T 00000000000000000000000000000000 P a7daa656c4dd455a8a2880b8230a716f [term 0 FOLLOWER]: Advancing to term 1
I20260809 15:40:10.605774 3431 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260809 15:40:10.606328 27231 raft_consensus.cc:504] T 00000000000000000000000000000000 P a7daa656c4dd455a8a2880b8230a716f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a7daa656c4dd455a8a2880b8230a716f" member_type: VOTER last_known_addr { host: "127.3.89.254" port: 43123 } }
I20260809 15:40:10.606647 27231 leader_election.cc:302] T 00000000000000000000000000000000 P a7daa656c4dd455a8a2880b8230a716f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: a7daa656c4dd455a8a2880b8230a716f; no voters:
I20260809 15:40:10.606997 27231 leader_election.cc:288] T 00000000000000000000000000000000 P a7daa656c4dd455a8a2880b8230a716f [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 15:40:10.607143 27238 raft_consensus.cc:2781] T 00000000000000000000000000000000 P a7daa656c4dd455a8a2880b8230a716f [term 1 FOLLOWER]: Leader election won for term 1
I20260809 15:40:10.607340 27238 raft_consensus.cc:686] T 00000000000000000000000000000000 P a7daa656c4dd455a8a2880b8230a716f [term 1 LEADER]: Becoming Leader. State: Replica: a7daa656c4dd455a8a2880b8230a716f, State: Running, Role: LEADER
I20260809 15:40:10.607689 27238 consensus_queue.cc:237] T 00000000000000000000000000000000 P a7daa656c4dd455a8a2880b8230a716f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a7daa656c4dd455a8a2880b8230a716f" member_type: VOTER last_known_addr { host: "127.3.89.254" port: 43123 } }
I20260809 15:40:10.608244 27231 sys_catalog.cc:564] T 00000000000000000000000000000000 P a7daa656c4dd455a8a2880b8230a716f [sys.catalog]: configured and running, proceeding with master startup.
I20260809 15:40:10.609220 27238 sys_catalog.cc:455] T 00000000000000000000000000000000 P a7daa656c4dd455a8a2880b8230a716f [sys.catalog]: SysCatalogTable state changed. Reason: New leader a7daa656c4dd455a8a2880b8230a716f. Latest consensus state: current_term: 1 leader_uuid: "a7daa656c4dd455a8a2880b8230a716f" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a7daa656c4dd455a8a2880b8230a716f" member_type: VOTER last_known_addr { host: "127.3.89.254" port: 43123 } } }
I20260809 15:40:10.609776 27243 sys_catalog.cc:455] T 00000000000000000000000000000000 P a7daa656c4dd455a8a2880b8230a716f [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "a7daa656c4dd455a8a2880b8230a716f" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a7daa656c4dd455a8a2880b8230a716f" member_type: VOTER last_known_addr { host: "127.3.89.254" port: 43123 } } }
I20260809 15:40:10.609954 27243 sys_catalog.cc:458] T 00000000000000000000000000000000 P a7daa656c4dd455a8a2880b8230a716f [sys.catalog]: This master's current role is: LEADER
I20260809 15:40:10.610252 27238 sys_catalog.cc:458] T 00000000000000000000000000000000 P a7daa656c4dd455a8a2880b8230a716f [sys.catalog]: This master's current role is: LEADER
I20260809 15:40:10.610365 27252 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 15:40:10.613521 27252 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 15:40:10.617831 27234 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:40:10.607906 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50089 (local address 127.3.89.254:43123)
0809 15:40:10.608035 (+ 129us) server_negotiation.cc:206] Beginning negotiation
0809 15:40:10.608039 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 15:40:10.608177 (+ 138us) server_negotiation.cc:407] Connection header received
0809 15:40:10.608294 (+ 117us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 15:40:10.608297 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 15:40:10.608337 (+ 40us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 15:40:10.608429 (+ 92us) server_negotiation.cc:226] Negotiated authn=SASL
0809 15:40:10.609229 (+ 800us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:40:10.610448 (+ 1219us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:40:10.612099 (+ 1651us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:40:10.612436 (+ 337us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:40:10.612552 (+ 116us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:40:10.614151 (+ 1599us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 15:40:10.614156 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 15:40:10.614159 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 15:40:10.614184 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0809 15:40:10.615970 (+ 1786us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 15:40:10.616499 (+ 529us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 15:40:10.616502 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 15:40:10.616504 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 15:40:10.616591 (+ 87us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 15:40:10.616912 (+ 321us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 15:40:10.616915 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 15:40:10.616917 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 15:40:10.617124 (+ 207us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 15:40:10.617266 (+ 142us) server_negotiation.cc:1035] Waiting for connection context
0809 15:40:10.617547 (+ 281us) server_negotiation.cc:299] Negotiation successful
0809 15:40:10.617664 (+ 117us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":66}
I20260809 15:40:10.619398 27252 catalog_manager.cc:1347] Generated new cluster ID: 28d79e1a786240418146d413a6aba2e8
I20260809 15:40:10.619470 27252 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 15:40:10.638082 27252 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 15:40:10.639278 27252 catalog_manager.cc:1504] Loading token signing keys...
I20260809 15:40:10.645898 27252 catalog_manager.cc:5868] T 00000000000000000000000000000000 P a7daa656c4dd455a8a2880b8230a716f: Generated new TSK 0
I20260809 15:40:10.646652 27252 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260809 15:40:10.657998 3431 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskm314YR/build/debug/bin/kudu with pid 27156
2026-08-09T15:40:10Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/15 (1449 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/16
Loading random data
Initializing database '/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786289732823074-3431-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 15:40:10 dist-test-slave-2w3w krb5kdc[27266](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 15:40:10 dist-test-slave-2w3w krb5kdc[27266](info): set up 2 sockets
Aug 09 15:40:10 dist-test-slave-2w3w krb5kdc[27266](info): commencing operation
krb5kdc: starting...
W20260809 15:40:11.737850 3431 mini_kdc.cc:121] Time spent starting KDC: real 1.058s user 0.000s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786289732823074-3431-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786289732823074-3431-0/krb5kdc/test-user.keytab.
Aug 09 15:40:11 dist-test-slave-2w3w krb5kdc[27266](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786290011, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T15:40:11Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T15:40:11Z Disabled control of system clock
WARNING: no policy specified for kudu/127.3.89.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.3.89.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.3.89.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786289732823074-3431-0/krb5kdc/kudu_127.3.89.254.keytab.
Entry for principal kudu/127.3.89.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786289732823074-3431-0/krb5kdc/kudu_127.3.89.254.keytab.
I20260809 15:40:11.863898 3431 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskm314YR/build/debug/bin/kudu
/tmp/dist-test-taskm314YR/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786289732823074-3431-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786289732823074-3431-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786289732823074-3431-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786289732823074-3431-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.3.89.254:46741
--webserver_interface=127.3.89.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786289732823074-3431-0/krb5kdc/kudu_127.3.89.254.keytab
--principal=kudu/127.3.89.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.3.89.212:38619
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.3.89.254:46741
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786289732823074-3431-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786289732823074-3431-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786289732823074-3431-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 15:40:11.973568 27281 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 15:40:11.973878 27281 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 15:40:11.973959 27281 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 15:40:11.978047 27281 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 15:40:11.978134 27281 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 15:40:11.978175 27281 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 15:40:11.978228 27281 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 15:40:11.978264 27281 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 15:40:11.983635 27281 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.3.89.212:38619
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786289732823074-3431-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786289732823074-3431-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.3.89.254:46741
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786289732823074-3431-0/krb5kdc/kudu_127.3.89.254.keytab
--principal=kudu/127.3.89.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.3.89.254:46741
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786289732823074-3431-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.3.89.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.27281
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786289732823074-3431-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 09 Aug 2026 14:46:55 UTC on ccycloud.quasar-yswkmf.root.comops.site
build id 4576540
I20260809 15:40:11.984810 27281 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 15:40:11.985831 27281 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 15:40:11.994117 27290 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 15:40:11.994181 27287 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 15:40:11.994222 27288 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 15:40:11.994426 27281 server_base.cc:1034] running on GCE node
I20260809 15:40:11.995122 27281 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 15:40:11.996212 27281 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 15:40:11.997394 27281 hybrid_clock.cc:648] HybridClock initialized: now 1786290011997374 us; error 28 us; skew 500 ppm
Aug 09 15:40:11 dist-test-slave-2w3w krb5kdc[27266](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786290011, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.89.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 15:40:12.000262 27281 init.cc:377] Logged in from keytab as kudu/127.3.89.254@KRBTEST.COM (short username kudu)
I20260809 15:40:12.001187 27281 webserver.cc:460] Webserver started at http://127.3.89.254:37247/ using document root <none> and password file <none>
I20260809 15:40:12.001752 27281 fs_manager.cc:359] Metadata directory not provided
I20260809 15:40:12.001816 27281 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 15:40:12.002055 27281 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 15:40:12.004201 27281 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskm314YR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786289732823074-3431-0/minicluster-data/master-0/data/instance:
uuid: "d617945924f3421598d055fafccfc556"
format_stamp: "Formatted at 2026-08-09 15:40:11 on dist-test-slave-2w3w"
server_key: "f6334a4baea180c751401fc16f059cef"
server_key_iv: "1654fc80df4b11df80d5ab2173f8316c"
server_key_version: "encryptionkey@0"
I20260809 15:40:12.004793 27281 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskm314YR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786289732823074-3431-0/minicluster-data/master-0/wal/instance:
uuid: "d617945924f3421598d055fafccfc556"
format_stamp: "Formatted at 2026-08-09 15:40:11 on dist-test-slave-2w3w"
server_key: "f6334a4baea180c751401fc16f059cef"
server_key_iv: "1654fc80df4b11df80d5ab2173f8316c"
server_key_version: "encryptionkey@0"
I20260809 15:40:12.008663 27281 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260809 15:40:12.011142 27297 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 15:40:12.012425 27281 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260809 15:40:12.012567 27281 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskm314YR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786289732823074-3431-0/minicluster-data/master-0/data,/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786289732823074-3431-0/minicluster-data/master-0/wal
uuid: "d617945924f3421598d055fafccfc556"
format_stamp: "Formatted at 2026-08-09 15:40:11 on dist-test-slave-2w3w"
server_key: "f6334a4baea180c751401fc16f059cef"
server_key_iv: "1654fc80df4b11df80d5ab2173f8316c"
server_key_version: "encryptionkey@0"
I20260809 15:40:12.012679 27281 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskm314YR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786289732823074-3431-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskm314YR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786289732823074-3431-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskm314YR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786289732823074-3431-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 15:40:12.031064 27281 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 15:40:12.033891 27281 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 15:40:12.034103 27281 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 15:40:12.041600 27281 rpc_server.cc:307] RPC server started. Bound to: 127.3.89.254:46741
I20260809 15:40:12.041630 27351 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.3.89.254:46741 every 8 connection(s)
I20260809 15:40:12.042511 27281 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskm314YR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786289732823074-3431-0/minicluster-data/master-0/data/info.pb
I20260809 15:40:12.045392 27352 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 15:40:12.049716 3431 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskm314YR/build/debug/bin/kudu as pid 27281
I20260809 15:40:12.049870 3431 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskm314YR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786289732823074-3431-0/minicluster-data/master-0/wal/instance
I20260809 15:40:12.050169 3431 external_mini_cluster.cc:1428] Setting key dc196061848baaed7b6a35eb452fb6c5
I20260809 15:40:12.052882 27352 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d617945924f3421598d055fafccfc556: Bootstrap starting.
I20260809 15:40:12.055155 27352 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P d617945924f3421598d055fafccfc556: Neither blocks nor log segments found. Creating new log.
I20260809 15:40:12.056027 27352 log.cc:824] T 00000000000000000000000000000000 P d617945924f3421598d055fafccfc556: Log is configured to *not* fsync() on all Append() calls
I20260809 15:40:12.058239 27352 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d617945924f3421598d055fafccfc556: No bootstrap required, opened a new log
I20260809 15:40:12.062232 27356 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:40:12.052064 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43271 (local address 127.3.89.254:46741)
0809 15:40:12.052907 (+ 843us) server_negotiation.cc:206] Beginning negotiation
0809 15:40:12.052924 (+ 17us) server_negotiation.cc:399] Waiting for connection header
0809 15:40:12.052978 (+ 54us) server_negotiation.cc:407] Connection header received
0809 15:40:12.053956 (+ 978us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 15:40:12.053997 (+ 41us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 15:40:12.054385 (+ 388us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 15:40:12.054677 (+ 292us) server_negotiation.cc:226] Negotiated authn=SASL
0809 15:40:12.055123 (+ 446us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:40:12.056701 (+ 1578us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:40:12.058017 (+ 1316us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:40:12.058803 (+ 786us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:40:12.059018 (+ 215us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:40:12.059907 (+ 889us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 15:40:12.059915 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 15:40:12.059934 (+ 19us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 15:40:12.060525 (+ 591us) server_negotiation.cc:922] calling sasl_server_start()
0809 15:40:12.060602 (+ 77us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 15:40:12.060640 (+ 38us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 15:40:12.060790 (+ 150us) server_negotiation.cc:1035] Waiting for connection context
0809 15:40:12.060988 (+ 198us) server_negotiation.cc:299] Negotiation successful
0809 15:40:12.061215 (+ 227us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":694,"thread_start_us":522,"threads_started":1}
I20260809 15:40:12.063143 27352 raft_consensus.cc:348] T 00000000000000000000000000000000 P d617945924f3421598d055fafccfc556 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d617945924f3421598d055fafccfc556" member_type: VOTER last_known_addr { host: "127.3.89.254" port: 46741 } }
I20260809 15:40:12.063490 27352 raft_consensus.cc:374] T 00000000000000000000000000000000 P d617945924f3421598d055fafccfc556 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 15:40:12.063560 27352 raft_consensus.cc:729] T 00000000000000000000000000000000 P d617945924f3421598d055fafccfc556 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d617945924f3421598d055fafccfc556, State: Initialized, Role: FOLLOWER
I20260809 15:40:12.064073 27352 consensus_queue.cc:260] T 00000000000000000000000000000000 P d617945924f3421598d055fafccfc556 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d617945924f3421598d055fafccfc556" member_type: VOTER last_known_addr { host: "127.3.89.254" port: 46741 } }
I20260809 15:40:12.064252 27352 raft_consensus.cc:388] T 00000000000000000000000000000000 P d617945924f3421598d055fafccfc556 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 15:40:12.064333 27352 raft_consensus.cc:482] T 00000000000000000000000000000000 P d617945924f3421598d055fafccfc556 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 15:40:12.064422 27352 raft_consensus.cc:3037] T 00000000000000000000000000000000 P d617945924f3421598d055fafccfc556 [term 0 FOLLOWER]: Advancing to term 1
I20260809 15:40:12.065327 3431 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260809 15:40:12.065805 27352 raft_consensus.cc:504] T 00000000000000000000000000000000 P d617945924f3421598d055fafccfc556 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d617945924f3421598d055fafccfc556" member_type: VOTER last_known_addr { host: "127.3.89.254" port: 46741 } }
I20260809 15:40:12.066184 27352 leader_election.cc:302] T 00000000000000000000000000000000 P d617945924f3421598d055fafccfc556 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: d617945924f3421598d055fafccfc556; no voters:
I20260809 15:40:12.066527 27352 leader_election.cc:288] T 00000000000000000000000000000000 P d617945924f3421598d055fafccfc556 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 15:40:12.066620 27358 raft_consensus.cc:2781] T 00000000000000000000000000000000 P d617945924f3421598d055fafccfc556 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 15:40:12.066833 27358 raft_consensus.cc:686] T 00000000000000000000000000000000 P d617945924f3421598d055fafccfc556 [term 1 LEADER]: Becoming Leader. State: Replica: d617945924f3421598d055fafccfc556, State: Running, Role: LEADER
I20260809 15:40:12.067149 27358 consensus_queue.cc:237] T 00000000000000000000000000000000 P d617945924f3421598d055fafccfc556 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d617945924f3421598d055fafccfc556" member_type: VOTER last_known_addr { host: "127.3.89.254" port: 46741 } }
I20260809 15:40:12.067799 27352 sys_catalog.cc:564] T 00000000000000000000000000000000 P d617945924f3421598d055fafccfc556 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 15:40:12.068655 27361 sys_catalog.cc:455] T 00000000000000000000000000000000 P d617945924f3421598d055fafccfc556 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "d617945924f3421598d055fafccfc556" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d617945924f3421598d055fafccfc556" member_type: VOTER last_known_addr { host: "127.3.89.254" port: 46741 } } }
I20260809 15:40:12.068819 27361 sys_catalog.cc:458] T 00000000000000000000000000000000 P d617945924f3421598d055fafccfc556 [sys.catalog]: This master's current role is: LEADER
I20260809 15:40:12.069152 27362 sys_catalog.cc:455] T 00000000000000000000000000000000 P d617945924f3421598d055fafccfc556 [sys.catalog]: SysCatalogTable state changed. Reason: New leader d617945924f3421598d055fafccfc556. Latest consensus state: current_term: 1 leader_uuid: "d617945924f3421598d055fafccfc556" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d617945924f3421598d055fafccfc556" member_type: VOTER last_known_addr { host: "127.3.89.254" port: 46741 } } }
I20260809 15:40:12.069262 27362 sys_catalog.cc:458] T 00000000000000000000000000000000 P d617945924f3421598d055fafccfc556 [sys.catalog]: This master's current role is: LEADER
I20260809 15:40:12.070127 27371 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 15:40:12.073226 27371 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 15:40:12.075012 27356 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:40:12.067897 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52835 (local address 127.3.89.254:46741)
0809 15:40:12.068043 (+ 146us) server_negotiation.cc:206] Beginning negotiation
0809 15:40:12.068048 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 15:40:12.070482 (+ 2434us) server_negotiation.cc:407] Connection header received
0809 15:40:12.070602 (+ 120us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 15:40:12.070605 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 15:40:12.070661 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 15:40:12.070746 (+ 85us) server_negotiation.cc:226] Negotiated authn=SASL
0809 15:40:12.071294 (+ 548us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:40:12.072075 (+ 781us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:40:12.073179 (+ 1104us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:40:12.073501 (+ 322us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:40:12.073615 (+ 114us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:40:12.074471 (+ 856us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 15:40:12.074475 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 15:40:12.074477 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 15:40:12.074490 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0809 15:40:12.074520 (+ 30us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 15:40:12.074537 (+ 17us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 15:40:12.074622 (+ 85us) server_negotiation.cc:1035] Waiting for connection context
0809 15:40:12.074743 (+ 121us) server_negotiation.cc:299] Negotiation successful
0809 15:40:12.074852 (+ 109us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":45}
I20260809 15:40:12.080976 27371 catalog_manager.cc:1347] Generated new cluster ID: 2d4b64dc980c4ab884f3f037d57e085c
I20260809 15:40:12.081040 27371 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 15:40:12.089116 27371 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 15:40:12.090339 27371 catalog_manager.cc:1504] Loading token signing keys...
I20260809 15:40:12.109259 27371 catalog_manager.cc:5868] T 00000000000000000000000000000000 P d617945924f3421598d055fafccfc556: Generated new TSK 0
I20260809 15:40:12.110100 27371 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260809 15:40:12.116904 3431 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskm314YR/build/debug/bin/kudu with pid 27281
2026-08-09T15:40:12Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/16 (1458 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/17
Loading random data
Initializing database '/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786289732823074-3431-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 15:40:12 dist-test-slave-2w3w krb5kdc[27384](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 15:40:12 dist-test-slave-2w3w krb5kdc[27384](info): set up 2 sockets
Aug 09 15:40:12 dist-test-slave-2w3w krb5kdc[27384](info): commencing operation
krb5kdc: starting...
W20260809 15:40:13.189604 3431 mini_kdc.cc:121] Time spent starting KDC: real 1.051s user 0.000s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786289732823074-3431-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786289732823074-3431-0/krb5kdc/test-user.keytab.
Aug 09 15:40:13 dist-test-slave-2w3w krb5kdc[27384](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786290013, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T15:40:13Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T15:40:13Z Disabled control of system clock
WARNING: no policy specified for kudu/127.3.89.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.3.89.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.3.89.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786289732823074-3431-0/krb5kdc/kudu_127.3.89.254.keytab.
Entry for principal kudu/127.3.89.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786289732823074-3431-0/krb5kdc/kudu_127.3.89.254.keytab.
I20260809 15:40:13.317791 3431 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskm314YR/build/debug/bin/kudu
/tmp/dist-test-taskm314YR/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786289732823074-3431-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786289732823074-3431-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786289732823074-3431-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786289732823074-3431-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.3.89.254:40951
--webserver_interface=127.3.89.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786289732823074-3431-0/krb5kdc/kudu_127.3.89.254.keytab
--principal=kudu/127.3.89.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.3.89.212:44931
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.3.89.254:40951
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786289732823074-3431-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786289732823074-3431-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786289732823074-3431-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 15:40:13.426306 27400 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 15:40:13.426618 27400 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 15:40:13.426693 27400 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 15:40:13.430665 27400 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 15:40:13.430756 27400 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 15:40:13.430794 27400 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 15:40:13.430840 27400 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 15:40:13.430874 27400 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 15:40:13.436025 27400 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.3.89.212:44931
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786289732823074-3431-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786289732823074-3431-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.3.89.254:40951
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786289732823074-3431-0/krb5kdc/kudu_127.3.89.254.keytab
--principal=kudu/127.3.89.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.3.89.254:40951
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786289732823074-3431-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.3.89.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.27400
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786289732823074-3431-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 09 Aug 2026 14:46:55 UTC on ccycloud.quasar-yswkmf.root.comops.site
build id 4576540
I20260809 15:40:13.437057 27400 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 15:40:13.438102 27400 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 15:40:13.446364 27408 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 15:40:13.446375 27405 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 15:40:13.446501 27400 server_base.cc:1034] running on GCE node
W20260809 15:40:13.446386 27406 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 15:40:13.447269 27400 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 15:40:13.448228 27400 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 15:40:13.449381 27400 hybrid_clock.cc:648] HybridClock initialized: now 1786290013449350 us; error 44 us; skew 500 ppm
Aug 09 15:40:13 dist-test-slave-2w3w krb5kdc[27384](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786290013, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.89.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 15:40:13.455848 27400 init.cc:377] Logged in from keytab as kudu/127.3.89.254@KRBTEST.COM (short username kudu)
I20260809 15:40:13.456737 27400 webserver.cc:460] Webserver started at http://127.3.89.254:40693/ using document root <none> and password file <none>
I20260809 15:40:13.457275 27400 fs_manager.cc:359] Metadata directory not provided
I20260809 15:40:13.457365 27400 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 15:40:13.457587 27400 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 15:40:13.459755 27400 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskm314YR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786289732823074-3431-0/minicluster-data/master-0/data/instance:
uuid: "e8f609dc7087453d8b1bc2432e9129bf"
format_stamp: "Formatted at 2026-08-09 15:40:13 on dist-test-slave-2w3w"
server_key: "e8100f652bab3e74ebb476e2d8c062c2"
server_key_iv: "a8ce7cd683a522349a2098fdb52f821a"
server_key_version: "encryptionkey@0"
I20260809 15:40:13.460337 27400 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskm314YR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786289732823074-3431-0/minicluster-data/master-0/wal/instance:
uuid: "e8f609dc7087453d8b1bc2432e9129bf"
format_stamp: "Formatted at 2026-08-09 15:40:13 on dist-test-slave-2w3w"
server_key: "e8100f652bab3e74ebb476e2d8c062c2"
server_key_iv: "a8ce7cd683a522349a2098fdb52f821a"
server_key_version: "encryptionkey@0"
I20260809 15:40:13.463944 27400 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.004s sys 0.000s
I20260809 15:40:13.466187 27416 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 15:40:13.467326 27400 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260809 15:40:13.467463 27400 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskm314YR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786289732823074-3431-0/minicluster-data/master-0/data,/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786289732823074-3431-0/minicluster-data/master-0/wal
uuid: "e8f609dc7087453d8b1bc2432e9129bf"
format_stamp: "Formatted at 2026-08-09 15:40:13 on dist-test-slave-2w3w"
server_key: "e8100f652bab3e74ebb476e2d8c062c2"
server_key_iv: "a8ce7cd683a522349a2098fdb52f821a"
server_key_version: "encryptionkey@0"
I20260809 15:40:13.467566 27400 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskm314YR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786289732823074-3431-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskm314YR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786289732823074-3431-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskm314YR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786289732823074-3431-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 15:40:13.471892 27400 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 15:40:13.474581 27400 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 15:40:13.474773 27400 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 15:40:13.481837 27400 rpc_server.cc:307] RPC server started. Bound to: 127.3.89.254:40951
I20260809 15:40:13.481863 27472 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.3.89.254:40951 every 8 connection(s)
I20260809 15:40:13.482712 27400 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskm314YR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786289732823074-3431-0/minicluster-data/master-0/data/info.pb
I20260809 15:40:13.483618 3431 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskm314YR/build/debug/bin/kudu as pid 27400
I20260809 15:40:13.483773 3431 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskm314YR/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786289732823074-3431-0/minicluster-data/master-0/wal/instance
I20260809 15:40:13.484064 3431 external_mini_cluster.cc:1428] Setting key c23a254f0181145ec19e5cc8f2ea48e8
I20260809 15:40:13.485945 27474 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 15:40:13.489055 27477 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:40:13.485475 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41191 (local address 127.3.89.254:40951)
0809 15:40:13.485910 (+ 435us) server_negotiation.cc:206] Beginning negotiation
0809 15:40:13.485923 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0809 15:40:13.485977 (+ 54us) server_negotiation.cc:407] Connection header received
0809 15:40:13.486631 (+ 654us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 15:40:13.486646 (+ 15us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 15:40:13.486913 (+ 267us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 15:40:13.487289 (+ 376us) server_negotiation.cc:226] Negotiated authn=SASL
0809 15:40:13.487977 (+ 688us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 15:40:13.487982 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 15:40:13.487992 (+ 10us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 15:40:13.488001 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0809 15:40:13.488045 (+ 44us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 15:40:13.488061 (+ 16us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 15:40:13.488204 (+ 143us) server_negotiation.cc:1035] Waiting for connection context
0809 15:40:13.488269 (+ 65us) server_negotiation.cc:299] Negotiation successful
0809 15:40:13.488409 (+ 140us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":285,"thread_start_us":144,"threads_started":1}
I20260809 15:40:13.491906 3431 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260809 15:40:13.492322 27474 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e8f609dc7087453d8b1bc2432e9129bf: Bootstrap starting.
I20260809 15:40:13.495051 27474 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P e8f609dc7087453d8b1bc2432e9129bf: Neither blocks nor log segments found. Creating new log.
I20260809 15:40:13.495831 27477 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:40:13.494160 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53505 (local address 127.3.89.254:40951)
0809 15:40:13.494281 (+ 121us) server_negotiation.cc:206] Beginning negotiation
0809 15:40:13.494286 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 15:40:13.494468 (+ 182us) server_negotiation.cc:407] Connection header received
0809 15:40:13.494575 (+ 107us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 15:40:13.494577 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 15:40:13.494604 (+ 27us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 15:40:13.494666 (+ 62us) server_negotiation.cc:226] Negotiated authn=SASL
0809 15:40:13.495439 (+ 773us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 15:40:13.495442 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 15:40:13.495443 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 15:40:13.495453 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0809 15:40:13.495477 (+ 24us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 15:40:13.495486 (+ 9us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 15:40:13.495553 (+ 67us) server_negotiation.cc:1035] Waiting for connection context
0809 15:40:13.495660 (+ 107us) server_negotiation.cc:299] Negotiation successful
0809 15:40:13.495690 (+ 30us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":48}
I20260809 15:40:13.495988 27474 log.cc:824] T 00000000000000000000000000000000 P e8f609dc7087453d8b1bc2432e9129bf: Log is configured to *not* fsync() on all Append() calls
I20260809 15:40:13.497769 27474 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e8f609dc7087453d8b1bc2432e9129bf: No bootstrap required, opened a new log
I20260809 15:40:13.501181 27474 raft_consensus.cc:348] T 00000000000000000000000000000000 P e8f609dc7087453d8b1bc2432e9129bf [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e8f609dc7087453d8b1bc2432e9129bf" member_type: VOTER last_known_addr { host: "127.3.89.254" port: 40951 } }
I20260809 15:40:13.501406 27474 raft_consensus.cc:374] T 00000000000000000000000000000000 P e8f609dc7087453d8b1bc2432e9129bf [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 15:40:13.501482 27474 raft_consensus.cc:729] T 00000000000000000000000000000000 P e8f609dc7087453d8b1bc2432e9129bf [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e8f609dc7087453d8b1bc2432e9129bf, State: Initialized, Role: FOLLOWER
I20260809 15:40:13.501876 27474 consensus_queue.cc:260] T 00000000000000000000000000000000 P e8f609dc7087453d8b1bc2432e9129bf [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e8f609dc7087453d8b1bc2432e9129bf" member_type: VOTER last_known_addr { host: "127.3.89.254" port: 40951 } }
I20260809 15:40:13.501991 27474 raft_consensus.cc:388] T 00000000000000000000000000000000 P e8f609dc7087453d8b1bc2432e9129bf [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 15:40:13.502074 27474 raft_consensus.cc:482] T 00000000000000000000000000000000 P e8f609dc7087453d8b1bc2432e9129bf [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 15:40:13.502193 27474 raft_consensus.cc:3037] T 00000000000000000000000000000000 P e8f609dc7087453d8b1bc2432e9129bf [term 0 FOLLOWER]: Advancing to term 1
I20260809 15:40:13.503211 27474 raft_consensus.cc:504] T 00000000000000000000000000000000 P e8f609dc7087453d8b1bc2432e9129bf [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e8f609dc7087453d8b1bc2432e9129bf" member_type: VOTER last_known_addr { host: "127.3.89.254" port: 40951 } }
I20260809 15:40:13.503536 27474 leader_election.cc:302] T 00000000000000000000000000000000 P e8f609dc7087453d8b1bc2432e9129bf [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: e8f609dc7087453d8b1bc2432e9129bf; no voters:
I20260809 15:40:13.503824 27474 leader_election.cc:288] T 00000000000000000000000000000000 P e8f609dc7087453d8b1bc2432e9129bf [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 15:40:13.504006 27485 raft_consensus.cc:2781] T 00000000000000000000000000000000 P e8f609dc7087453d8b1bc2432e9129bf [term 1 FOLLOWER]: Leader election won for term 1
I20260809 15:40:13.504262 27485 raft_consensus.cc:686] T 00000000000000000000000000000000 P e8f609dc7087453d8b1bc2432e9129bf [term 1 LEADER]: Becoming Leader. State: Replica: e8f609dc7087453d8b1bc2432e9129bf, State: Running, Role: LEADER
I20260809 15:40:13.504611 27485 consensus_queue.cc:237] T 00000000000000000000000000000000 P e8f609dc7087453d8b1bc2432e9129bf [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e8f609dc7087453d8b1bc2432e9129bf" member_type: VOTER last_known_addr { host: "127.3.89.254" port: 40951 } }
I20260809 15:40:13.504993 27474 sys_catalog.cc:564] T 00000000000000000000000000000000 P e8f609dc7087453d8b1bc2432e9129bf [sys.catalog]: configured and running, proceeding with master startup.
I20260809 15:40:13.506318 27487 sys_catalog.cc:455] T 00000000000000000000000000000000 P e8f609dc7087453d8b1bc2432e9129bf [sys.catalog]: SysCatalogTable state changed. Reason: New leader e8f609dc7087453d8b1bc2432e9129bf. Latest consensus state: current_term: 1 leader_uuid: "e8f609dc7087453d8b1bc2432e9129bf" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e8f609dc7087453d8b1bc2432e9129bf" member_type: VOTER last_known_addr { host: "127.3.89.254" port: 40951 } } }
I20260809 15:40:13.506371 27486 sys_catalog.cc:455] T 00000000000000000000000000000000 P e8f609dc7087453d8b1bc2432e9129bf [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "e8f609dc7087453d8b1bc2432e9129bf" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e8f609dc7087453d8b1bc2432e9129bf" member_type: VOTER last_known_addr { host: "127.3.89.254" port: 40951 } } }
I20260809 15:40:13.506435 27487 sys_catalog.cc:458] T 00000000000000000000000000000000 P e8f609dc7087453d8b1bc2432e9129bf [sys.catalog]: This master's current role is: LEADER
I20260809 15:40:13.506474 27486 sys_catalog.cc:458] T 00000000000000000000000000000000 P e8f609dc7087453d8b1bc2432e9129bf [sys.catalog]: This master's current role is: LEADER
I20260809 15:40:13.506989 27495 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 15:40:13.510079 27495 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 15:40:13.515264 27495 catalog_manager.cc:1347] Generated new cluster ID: 49ade79bf410441eb2ad3762630d7bdb
I20260809 15:40:13.515332 27495 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 15:40:13.530054 27495 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 15:40:13.530369 27495 catalog_manager.cc:1504] Loading token signing keys...
I20260809 15:40:13.535952 27495 catalog_manager.cc:5868] T 00000000000000000000000000000000 P e8f609dc7087453d8b1bc2432e9129bf: Generated new TSK 0
I20260809 15:40:13.536644 27495 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260809 15:40:13.547896 3431 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskm314YR/build/debug/bin/kudu with pid 27400
2026-08-09T15:40:13Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/17 (1431 ms)
[----------] 18 tests from AuthTokenIssuingTest (26371 ms total)
[----------] 2 tests from ConnectToFollowerMasterTest
[ RUN ] ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys/0
Loading random data
Initializing database '/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786289732823074-3431-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 15:40:13 dist-test-slave-2w3w krb5kdc[27510](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 15:40:13 dist-test-slave-2w3w krb5kdc[27510](info): set up 2 sockets
Aug 09 15:40:13 dist-test-slave-2w3w krb5kdc[27510](info): commencing operation
krb5kdc: starting...
W20260809 15:40:14.636431 3431 mini_kdc.cc:121] Time spent starting KDC: real 1.067s user 0.000s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786289732823074-3431-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786289732823074-3431-0/krb5kdc/test-user.keytab.
Aug 09 15:40:14 dist-test-slave-2w3w krb5kdc[27510](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786290014, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T15:40:14Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T15:40:14Z Disabled control of system clock
WARNING: no policy specified for kudu/127.3.89.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.3.89.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.3.89.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786289732823074-3431-0/krb5kdc/kudu_127.3.89.254.keytab.
Entry for principal kudu/127.3.89.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786289732823074-3431-0/krb5kdc/kudu_127.3.89.254.keytab.
I20260809 15:40:14.759603 3431 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskm314YR/build/debug/bin/kudu
/tmp/dist-test-taskm314YR/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786289732823074-3431-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786289732823074-3431-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786289732823074-3431-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786289732823074-3431-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.3.89.254:33105
--webserver_interface=127.3.89.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786289732823074-3431-0/krb5kdc/kudu_127.3.89.254.keytab
--principal=kudu/127.3.89.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.3.89.212:37439
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.3.89.254:33105,127.3.89.253:37127,127.3.89.252:35283
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=optional with env {KRB5CCNAME=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786289732823074-3431-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786289732823074-3431-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786289732823074-3431-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 15:40:14.869426 27523 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260809 15:40:14.869763 27523 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 15:40:14.869827 27523 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 15:40:14.869904 27523 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 15:40:14.874074 27523 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 15:40:14.874162 27523 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 15:40:14.874203 27523 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 15:40:14.874264 27523 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 15:40:14.874303 27523 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 15:40:14.879616 27523 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.3.89.212:37439
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786289732823074-3431-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786289732823074-3431-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.3.89.254:33105,127.3.89.253:37127,127.3.89.252:35283
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786289732823074-3431-0/krb5kdc/kudu_127.3.89.254.keytab
--principal=kudu/127.3.89.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.3.89.254:33105
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786289732823074-3431-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.3.89.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.27523
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786289732823074-3431-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 09 Aug 2026 14:46:55 UTC on ccycloud.quasar-yswkmf.root.comops.site
build id 4576540
I20260809 15:40:14.880676 27523 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 15:40:14.881703 27523 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 15:40:14.890373 27532 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 15:40:14.890448 27534 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 15:40:14.890412 27531 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 15:40:14.890554 27523 server_base.cc:1034] running on GCE node
I20260809 15:40:14.891238 27523 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 15:40:14.892275 27523 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 15:40:14.893508 27523 hybrid_clock.cc:648] HybridClock initialized: now 1786290014893497 us; error 27 us; skew 500 ppm
Aug 09 15:40:14 dist-test-slave-2w3w krb5kdc[27510](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786290014, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.89.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 15:40:14.896407 27523 init.cc:377] Logged in from keytab as kudu/127.3.89.254@KRBTEST.COM (short username kudu)
I20260809 15:40:14.897341 27523 webserver.cc:460] Webserver started at http://127.3.89.254:44685/ using document root <none> and password file <none>
I20260809 15:40:14.897892 27523 fs_manager.cc:359] Metadata directory not provided
I20260809 15:40:14.897993 27523 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 15:40:14.898224 27523 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 15:40:14.900360 27523 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskm314YR/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786289732823074-3431-0/minicluster-data/master-0/data/instance:
uuid: "76ee3afa6e0e4189a630938d1e316744"
format_stamp: "Formatted at 2026-08-09 15:40:14 on dist-test-slave-2w3w"
server_key: "9714bfd66ba0a21a41bcd78e7d87bfac"
server_key_iv: "c087f6a178f37a343c49c771e6257bb2"
server_key_version: "encryptionkey@0"
I20260809 15:40:14.900943 27523 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskm314YR/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786289732823074-3431-0/minicluster-data/master-0/wal/instance:
uuid: "76ee3afa6e0e4189a630938d1e316744"
format_stamp: "Formatted at 2026-08-09 15:40:14 on dist-test-slave-2w3w"
server_key: "9714bfd66ba0a21a41bcd78e7d87bfac"
server_key_iv: "c087f6a178f37a343c49c771e6257bb2"
server_key_version: "encryptionkey@0"
I20260809 15:40:14.904644 27523 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.005s sys 0.000s
I20260809 15:40:14.907090 27541 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 15:40:14.908212 27523 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260809 15:40:14.908352 27523 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskm314YR/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786289732823074-3431-0/minicluster-data/master-0/data,/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786289732823074-3431-0/minicluster-data/master-0/wal
uuid: "76ee3afa6e0e4189a630938d1e316744"
format_stamp: "Formatted at 2026-08-09 15:40:14 on dist-test-slave-2w3w"
server_key: "9714bfd66ba0a21a41bcd78e7d87bfac"
server_key_iv: "c087f6a178f37a343c49c771e6257bb2"
server_key_version: "encryptionkey@0"
I20260809 15:40:14.908463 27523 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskm314YR/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786289732823074-3431-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskm314YR/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786289732823074-3431-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskm314YR/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786289732823074-3431-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 15:40:14.921838 27523 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 15:40:14.924485 27523 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 15:40:14.924682 27523 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 15:40:14.931983 27523 rpc_server.cc:307] RPC server started. Bound to: 127.3.89.254:33105
I20260809 15:40:14.932268 27596 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.3.89.254:33105 every 8 connection(s)
I20260809 15:40:14.932885 27523 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskm314YR/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786289732823074-3431-0/minicluster-data/master-0/data/info.pb
I20260809 15:40:14.935585 27597 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 15:40:14.936156 3431 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskm314YR/build/debug/bin/kudu as pid 27523
I20260809 15:40:14.936307 3431 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskm314YR/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786289732823074-3431-0/minicluster-data/master-0/wal/instance
I20260809 15:40:14.936601 3431 external_mini_cluster.cc:1428] Setting key bd3e95fc418a88306b96fda457ad9586
I20260809 15:40:14.939323 27597 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.3.89.254" port: 33105 } has no permanent_uuid. Determining permanent_uuid...
WARNING: no policy specified for kudu/127.3.89.253@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.3.89.253@KRBTEST.COM" created.
Aug 09 15:40:14 dist-test-slave-2w3w krb5kdc[27510](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786290014, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.89.254@KRBTEST.COM for kudu/127.3.89.254@KRBTEST.COM
I20260809 15:40:14.955363 27600 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:40:14.941500 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45250 (local address 127.3.89.254:33105)
0809 15:40:14.941902 (+ 402us) server_negotiation.cc:206] Beginning negotiation
0809 15:40:14.941916 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0809 15:40:14.942133 (+ 217us) server_negotiation.cc:407] Connection header received
0809 15:40:14.943227 (+ 1094us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 15:40:14.943237 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 15:40:14.943497 (+ 260us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 15:40:14.943709 (+ 212us) server_negotiation.cc:226] Negotiated authn=SASL
0809 15:40:14.944442 (+ 733us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:40:14.945904 (+ 1462us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:40:14.946903 (+ 999us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:40:14.947390 (+ 487us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:40:14.947517 (+ 127us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:40:14.950244 (+ 2727us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 15:40:14.950249 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 15:40:14.950258 (+ 9us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 15:40:14.950278 (+ 20us) server_negotiation.cc:922] calling sasl_server_start()
0809 15:40:14.952323 (+ 2045us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 15:40:14.952820 (+ 497us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 15:40:14.952828 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 15:40:14.952833 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0809 15:40:14.952976 (+ 143us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 15:40:14.953274 (+ 298us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 15:40:14.953281 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 15:40:14.953285 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 15:40:14.953835 (+ 550us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 15:40:14.953985 (+ 150us) server_negotiation.cc:1035] Waiting for connection context
0809 15:40:14.954344 (+ 359us) server_negotiation.cc:299] Negotiation successful
0809 15:40:14.954600 (+ 256us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":40,"server-negotiator.queue_time_us":264,"thread_start_us":132,"threads_started":1}
I20260809 15:40:14.955361 27602 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:40:14.941499 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.89.254:33105 (local address 127.0.0.1:45250)
0809 15:40:14.941978 (+ 479us) negotiation.cc:107] Waiting for socket to connect
0809 15:40:14.942027 (+ 49us) client_negotiation.cc:174] Beginning negotiation
0809 15:40:14.942856 (+ 829us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 15:40:14.943706 (+ 850us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 15:40:14.943710 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 15:40:14.944041 (+ 331us) client_negotiation.cc:189] Negotiated authn=SASL
0809 15:40:14.944278 (+ 237us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:40:14.944290 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:40:14.946029 (+ 1739us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:40:14.946032 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:40:14.946790 (+ 758us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:40:14.946799 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:40:14.947531 (+ 732us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:40:14.947534 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:40:14.947627 (+ 93us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:40:14.948203 (+ 576us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 15:40:14.948243 (+ 40us) client_negotiation.cc:656] calling sasl_client_start()
0809 15:40:14.950099 (+ 1856us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 15:40:14.952453 (+ 2354us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 15:40:14.952468 (+ 15us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 15:40:14.952476 (+ 8us) client_negotiation.cc:760] calling sasl_client_step()
0809 15:40:14.952704 (+ 228us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 15:40:14.953073 (+ 369us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 15:40:14.953075 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 15:40:14.953076 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0809 15:40:14.953182 (+ 106us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 15:40:14.953960 (+ 778us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 15:40:14.953967 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 15:40:14.954088 (+ 121us) client_negotiation.cc:769] Sending connection context
0809 15:40:14.954229 (+ 141us) client_negotiation.cc:240] Negotiation successful
0809 15:40:14.954488 (+ 259us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":299,"spinlock_wait_cycles":36736,"thread_start_us":164,"threads_started":1}
I20260809 15:40:14.957713 27597 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.3.89.253" port: 37127 } has no permanent_uuid. Determining permanent_uuid...
I20260809 15:40:14.958439 27602 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:40:14.958160 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.89.253:37127 (local address 127.0.0.1:47268)
0809 15:40:14.958300 (+ 140us) negotiation.cc:107] Waiting for socket to connect
0809 15:40:14.958380 (+ 80us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.89.253:37127: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":53}
W20260809 15:40:14.958623 27543 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.3.89.253:37127: connect: Connection refused (error 111)
I20260809 15:40:14.959548 27602 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:40:14.959280 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.89.253:37127 (local address 127.0.0.1:47280)
0809 15:40:14.959406 (+ 126us) negotiation.cc:107] Waiting for socket to connect
0809 15:40:14.959468 (+ 62us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.89.253:37127: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":41}
W20260809 15:40:14.959699 27597 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.3.89.253:37127: Network error: Client connection negotiation failed: client connection to 127.3.89.253:37127: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.3.89.253 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786289732823074-3431-0/krb5kdc/kudu_127.3.89.253.keytab.
Entry for principal kudu/127.3.89.253 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786289732823074-3431-0/krb5kdc/kudu_127.3.89.253.keytab.
I20260809 15:40:14.966202 3431 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskm314YR/build/debug/bin/kudu
/tmp/dist-test-taskm314YR/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786289732823074-3431-0/minicluster-data/master-1/wal
--fs_data_dirs=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786289732823074-3431-0/minicluster-data/master-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786289732823074-3431-0/minicluster-data/master-1/logs
--server_dump_info_path=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786289732823074-3431-0/minicluster-data/master-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.3.89.253:37127
--webserver_interface=127.3.89.253
--webserver_port=0
--keytab_file=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786289732823074-3431-0/krb5kdc/kudu_127.3.89.253.keytab
--principal=kudu/127.3.89.253
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.3.89.212:37439
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.3.89.254:33105,127.3.89.253:37127,127.3.89.252:35283
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=optional with env {KRB5CCNAME=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786289732823074-3431-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786289732823074-3431-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786289732823074-3431-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260809 15:40:15.009083 27597 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.3.89.253" port: 37127 } attempt: 1
I20260809 15:40:15.010083 27602 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:40:15.009770 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.89.253:37127 (local address 127.0.0.1:47296)
0809 15:40:15.009939 (+ 169us) negotiation.cc:107] Waiting for socket to connect
0809 15:40:15.010004 (+ 65us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.89.253:37127: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":69}
I20260809 15:40:15.010812 27602 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:40:15.010599 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.89.253:37127 (local address 127.0.0.1:47312)
0809 15:40:15.010706 (+ 107us) negotiation.cc:107] Waiting for socket to connect
0809 15:40:15.010756 (+ 50us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.89.253:37127: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":42}
W20260809 15:40:15.010939 27597 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.3.89.253:37127: Network error: Client connection negotiation failed: client connection to 127.3.89.253:37127: connect: Connection refused (error 111)
I20260809 15:40:15.079120 27597 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.3.89.253" port: 37127 } attempt: 2
I20260809 15:40:15.079994 27602 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:40:15.079673 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.89.253:37127 (local address 127.0.0.1:47316)
0809 15:40:15.079855 (+ 182us) negotiation.cc:107] Waiting for socket to connect
0809 15:40:15.079921 (+ 66us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.89.253:37127: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":90}
W20260809 15:40:15.080533 27605 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
I20260809 15:40:15.080732 27602 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:40:15.080512 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.89.253:37127 (local address 127.0.0.1:47330)
0809 15:40:15.080616 (+ 104us) negotiation.cc:107] Waiting for socket to connect
0809 15:40:15.080669 (+ 53us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.89.253:37127: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":41}
W20260809 15:40:15.080824 27605 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 15:40:15.080891 27597 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.3.89.253:37127: Network error: Client connection negotiation failed: client connection to 127.3.89.253:37127: connect: Connection refused (error 111)
W20260809 15:40:15.080911 27605 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 15:40:15.081000 27605 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 15:40:15.084985 27605 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 15:40:15.085033 27605 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 15:40:15.085052 27605 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 15:40:15.085069 27605 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 15:40:15.085084 27605 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 15:40:15.090562 27605 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.3.89.212:37439
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786289732823074-3431-0/minicluster-data/master-1/data
--fs_wal_dir=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786289732823074-3431-0/minicluster-data/master-1/wal
--ipki_ca_key_size=768
--master_addresses=127.3.89.254:33105,127.3.89.253:37127,127.3.89.252:35283
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786289732823074-3431-0/krb5kdc/kudu_127.3.89.253.keytab
--principal=kudu/127.3.89.253
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.3.89.253:37127
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786289732823074-3431-0/minicluster-data/master-1/data/info.pb
--webserver_interface=127.3.89.253
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.27605
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786289732823074-3431-0/minicluster-data/master-1/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 09 Aug 2026 14:46:55 UTC on ccycloud.quasar-yswkmf.root.comops.site
build id 4576540
I20260809 15:40:15.091688 27605 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 15:40:15.092653 27605 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 15:40:15.101164 27611 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 15:40:15.101338 27612 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 15:40:15.101182 27614 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 15:40:15.101655 27605 server_base.cc:1034] running on GCE node
I20260809 15:40:15.102298 27605 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 15:40:15.103521 27605 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 15:40:15.104712 27605 hybrid_clock.cc:648] HybridClock initialized: now 1786290015104685 us; error 37 us; skew 500 ppm
Aug 09 15:40:15 dist-test-slave-2w3w krb5kdc[27510](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786290015, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.89.253@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 15:40:15.107542 27605 init.cc:377] Logged in from keytab as kudu/127.3.89.253@KRBTEST.COM (short username kudu)
I20260809 15:40:15.108424 27605 webserver.cc:460] Webserver started at http://127.3.89.253:34055/ using document root <none> and password file <none>
I20260809 15:40:15.108994 27605 fs_manager.cc:359] Metadata directory not provided
I20260809 15:40:15.109086 27605 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 15:40:15.109333 27605 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 15:40:15.111460 27605 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskm314YR/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786289732823074-3431-0/minicluster-data/master-1/data/instance:
uuid: "1d43cc5aa2554fdf851c3663c9717f3c"
format_stamp: "Formatted at 2026-08-09 15:40:15 on dist-test-slave-2w3w"
server_key: "1a9c550e1fb1a647b0f74ae7ec0a4a83"
server_key_iv: "9d95d049f9ac8fedc8b31ab6ef28f841"
server_key_version: "encryptionkey@0"
I20260809 15:40:15.112111 27605 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskm314YR/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786289732823074-3431-0/minicluster-data/master-1/wal/instance:
uuid: "1d43cc5aa2554fdf851c3663c9717f3c"
format_stamp: "Formatted at 2026-08-09 15:40:15 on dist-test-slave-2w3w"
server_key: "1a9c550e1fb1a647b0f74ae7ec0a4a83"
server_key_iv: "9d95d049f9ac8fedc8b31ab6ef28f841"
server_key_version: "encryptionkey@0"
I20260809 15:40:15.116003 27605 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.001s
I20260809 15:40:15.118391 27621 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 15:40:15.119681 27605 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260809 15:40:15.119822 27605 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskm314YR/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786289732823074-3431-0/minicluster-data/master-1/data,/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786289732823074-3431-0/minicluster-data/master-1/wal
uuid: "1d43cc5aa2554fdf851c3663c9717f3c"
format_stamp: "Formatted at 2026-08-09 15:40:15 on dist-test-slave-2w3w"
server_key: "1a9c550e1fb1a647b0f74ae7ec0a4a83"
server_key_iv: "9d95d049f9ac8fedc8b31ab6ef28f841"
server_key_version: "encryptionkey@0"
I20260809 15:40:15.119943 27605 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskm314YR/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786289732823074-3431-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-taskm314YR/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786289732823074-3431-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-taskm314YR/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786289732823074-3431-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 15:40:15.148101 27605 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 15:40:15.150929 27605 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 15:40:15.151165 27605 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 15:40:15.158077 27605 rpc_server.cc:307] RPC server started. Bound to: 127.3.89.253:37127
I20260809 15:40:15.158305 27677 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.3.89.253:37127 every 8 connection(s)
I20260809 15:40:15.159081 27605 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskm314YR/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786289732823074-3431-0/minicluster-data/master-1/data/info.pb
I20260809 15:40:15.161724 27678 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 15:40:15.163548 3431 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskm314YR/build/debug/bin/kudu as pid 27605
I20260809 15:40:15.163672 3431 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskm314YR/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786289732823074-3431-0/minicluster-data/master-1/wal/instance
I20260809 15:40:15.163960 3431 external_mini_cluster.cc:1428] Setting key 30b67f24359b8c6d9add60cdc62060a9
I20260809 15:40:15.166491 27678 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.3.89.254" port: 33105 } has no permanent_uuid. Determining permanent_uuid...
I20260809 15:40:15.172081 27597 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.3.89.253" port: 37127 } attempt: 3
WARNING: no policy specified for kudu/127.3.89.252@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.3.89.252@KRBTEST.COM" created.
Aug 09 15:40:15 dist-test-slave-2w3w krb5kdc[27510](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786290015, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.89.253@KRBTEST.COM for kudu/127.3.89.254@KRBTEST.COM
Aug 09 15:40:15 dist-test-slave-2w3w krb5kdc[27510](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786290014, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.89.254@KRBTEST.COM for kudu/127.3.89.253@KRBTEST.COM
I20260809 15:40:15.183095 27602 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:40:15.172544 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.89.253:37127 (local address 127.0.0.1:47336)
0809 15:40:15.172673 (+ 129us) negotiation.cc:107] Waiting for socket to connect
0809 15:40:15.172691 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0809 15:40:15.172800 (+ 109us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 15:40:15.173521 (+ 721us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 15:40:15.173527 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 15:40:15.173743 (+ 216us) client_negotiation.cc:189] Negotiated authn=SASL
0809 15:40:15.173891 (+ 148us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:40:15.173898 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:40:15.175124 (+ 1226us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:40:15.175127 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:40:15.175892 (+ 765us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:40:15.175902 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:40:15.176627 (+ 725us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:40:15.176630 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:40:15.176714 (+ 84us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:40:15.177259 (+ 545us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 15:40:15.177290 (+ 31us) client_negotiation.cc:656] calling sasl_client_start()
0809 15:40:15.178730 (+ 1440us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 15:40:15.181050 (+ 2320us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 15:40:15.181055 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 15:40:15.181058 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0809 15:40:15.181273 (+ 215us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 15:40:15.181689 (+ 416us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 15:40:15.181693 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 15:40:15.181695 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 15:40:15.181780 (+ 85us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 15:40:15.182523 (+ 743us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 15:40:15.182526 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 15:40:15.182636 (+ 110us) client_negotiation.cc:769] Sending connection context
0809 15:40:15.182751 (+ 115us) client_negotiation.cc:240] Negotiation successful
0809 15:40:15.182891 (+ 140us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":39}
I20260809 15:40:15.184161 27683 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:40:15.172707 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47336 (local address 127.3.89.253:37127)
0809 15:40:15.173072 (+ 365us) server_negotiation.cc:206] Beginning negotiation
0809 15:40:15.173083 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0809 15:40:15.173104 (+ 21us) server_negotiation.cc:407] Connection header received
0809 15:40:15.173146 (+ 42us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 15:40:15.173151 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 15:40:15.173325 (+ 174us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 15:40:15.173491 (+ 166us) server_negotiation.cc:226] Negotiated authn=SASL
0809 15:40:15.174025 (+ 534us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:40:15.175022 (+ 997us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:40:15.176017 (+ 995us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:40:15.176534 (+ 517us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:40:15.176620 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:40:15.178898 (+ 2278us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 15:40:15.178906 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 15:40:15.178920 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 15:40:15.178951 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0809 15:40:15.180927 (+ 1976us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 15:40:15.181394 (+ 467us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 15:40:15.181401 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 15:40:15.181402 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 15:40:15.181585 (+ 183us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 15:40:15.181883 (+ 298us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 15:40:15.181886 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 15:40:15.181888 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 15:40:15.182400 (+ 512us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 15:40:15.182542 (+ 142us) server_negotiation.cc:1035] Waiting for connection context
0809 15:40:15.182900 (+ 358us) server_negotiation.cc:299] Negotiation successful
0809 15:40:15.183253 (+ 353us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":1,"server-negotiator.queue_time_us":271,"thread_start_us":176,"threads_started":1}
I20260809 15:40:15.184165 27682 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:40:15.168935 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.89.254:33105 (local address 127.0.0.1:45254)
0809 15:40:15.169448 (+ 513us) negotiation.cc:107] Waiting for socket to connect
0809 15:40:15.169503 (+ 55us) client_negotiation.cc:174] Beginning negotiation
0809 15:40:15.170371 (+ 868us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 15:40:15.170922 (+ 551us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 15:40:15.170929 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 15:40:15.171315 (+ 386us) client_negotiation.cc:189] Negotiated authn=SASL
0809 15:40:15.171780 (+ 465us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:40:15.171792 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:40:15.172784 (+ 992us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:40:15.172790 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:40:15.173927 (+ 1137us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:40:15.173936 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:40:15.174760 (+ 824us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:40:15.174764 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:40:15.175020 (+ 256us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:40:15.175672 (+ 652us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 15:40:15.175687 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0809 15:40:15.178029 (+ 2342us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 15:40:15.181365 (+ 3336us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 15:40:15.181379 (+ 14us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 15:40:15.181392 (+ 13us) client_negotiation.cc:760] calling sasl_client_step()
0809 15:40:15.181789 (+ 397us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 15:40:15.182194 (+ 405us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 15:40:15.182198 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 15:40:15.182200 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 15:40:15.182285 (+ 85us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 15:40:15.182840 (+ 555us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 15:40:15.182850 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 15:40:15.183032 (+ 182us) client_negotiation.cc:769] Sending connection context
0809 15:40:15.183189 (+ 157us) client_negotiation.cc:240] Negotiation successful
0809 15:40:15.183434 (+ 245us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":329,"mutex_wait_us":57,"thread_start_us":163,"threads_started":1}
I20260809 15:40:15.184728 27600 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:40:15.168420 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45254 (local address 127.3.89.254:33105)
0809 15:40:15.168560 (+ 140us) server_negotiation.cc:206] Beginning negotiation
0809 15:40:15.168563 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0809 15:40:15.169616 (+ 1053us) server_negotiation.cc:407] Connection header received
0809 15:40:15.170593 (+ 977us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 15:40:15.170596 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 15:40:15.170660 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 15:40:15.170760 (+ 100us) server_negotiation.cc:226] Negotiated authn=SASL
0809 15:40:15.171929 (+ 1169us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:40:15.172651 (+ 722us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:40:15.174131 (+ 1480us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:40:15.174603 (+ 472us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:40:15.174696 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:40:15.178186 (+ 3490us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 15:40:15.178193 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 15:40:15.178200 (+ 7us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 15:40:15.178235 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0809 15:40:15.181209 (+ 2974us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 15:40:15.181946 (+ 737us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 15:40:15.181952 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 15:40:15.181958 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0809 15:40:15.182054 (+ 96us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 15:40:15.182400 (+ 346us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 15:40:15.182407 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 15:40:15.182410 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 15:40:15.182687 (+ 277us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 15:40:15.184256 (+ 1569us) server_negotiation.cc:1035] Waiting for connection context
0809 15:40:15.184389 (+ 133us) server_negotiation.cc:299] Negotiation successful
0809 15:40:15.184531 (+ 142us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":3,"server-negotiator.queue_time_us":56}
I20260809 15:40:15.186060 27678 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.3.89.253" port: 37127 } has no permanent_uuid. Determining permanent_uuid...
I20260809 15:40:15.186625 27597 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.3.89.252" port: 35283 } has no permanent_uuid. Determining permanent_uuid...
I20260809 15:40:15.187608 27602 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:40:15.187352 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.89.252:35283 (local address 127.0.0.1:50220)
0809 15:40:15.187479 (+ 127us) negotiation.cc:107] Waiting for socket to connect
0809 15:40:15.187536 (+ 57us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.89.252:35283: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":51}
I20260809 15:40:15.188395 27602 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:40:15.188109 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.89.252:35283 (local address 127.0.0.1:50228)
0809 15:40:15.188251 (+ 142us) negotiation.cc:107] Waiting for socket to connect
0809 15:40:15.188327 (+ 76us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.89.252:35283: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":77}
W20260809 15:40:15.188534 27597 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.3.89.252:35283: Network error: Client connection negotiation failed: client connection to 127.3.89.252:35283: connect: Connection refused (error 111)
Aug 09 15:40:15 dist-test-slave-2w3w krb5kdc[27510](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786290015, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.89.253@KRBTEST.COM for kudu/127.3.89.253@KRBTEST.COM
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.3.89.252 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786289732823074-3431-0/krb5kdc/kudu_127.3.89.252.keytab.
Entry for principal kudu/127.3.89.252 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786289732823074-3431-0/krb5kdc/kudu_127.3.89.252.keytab.
I20260809 15:40:15.195372 27683 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:40:15.186862 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47340 (local address 127.3.89.253:37127)
0809 15:40:15.187571 (+ 709us) server_negotiation.cc:206] Beginning negotiation
0809 15:40:15.187575 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 15:40:15.187588 (+ 13us) server_negotiation.cc:407] Connection header received
0809 15:40:15.187623 (+ 35us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 15:40:15.187628 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 15:40:15.187677 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 15:40:15.187914 (+ 237us) server_negotiation.cc:226] Negotiated authn=SASL
0809 15:40:15.188376 (+ 462us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:40:15.188999 (+ 623us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:40:15.189832 (+ 833us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:40:15.190072 (+ 240us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:40:15.190155 (+ 83us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:40:15.192376 (+ 2221us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 15:40:15.192380 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 15:40:15.192382 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 15:40:15.192401 (+ 19us) server_negotiation.cc:922] calling sasl_server_start()
0809 15:40:15.193746 (+ 1345us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 15:40:15.194167 (+ 421us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 15:40:15.194172 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 15:40:15.194174 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 15:40:15.194251 (+ 77us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 15:40:15.194525 (+ 274us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 15:40:15.194528 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 15:40:15.194530 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 15:40:15.194696 (+ 166us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 15:40:15.194815 (+ 119us) server_negotiation.cc:1035] Waiting for connection context
0809 15:40:15.195099 (+ 284us) server_negotiation.cc:299] Negotiation successful
0809 15:40:15.195254 (+ 155us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":60,"server-negotiator.queue_time_us":634}
I20260809 15:40:15.195371 27682 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:40:15.186616 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.89.253:37127 (local address 127.0.0.1:47340)
0809 15:40:15.186735 (+ 119us) negotiation.cc:107] Waiting for socket to connect
0809 15:40:15.186750 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0809 15:40:15.186825 (+ 75us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 15:40:15.187958 (+ 1133us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 15:40:15.187962 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 15:40:15.188145 (+ 183us) client_negotiation.cc:189] Negotiated authn=SASL
0809 15:40:15.188240 (+ 95us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:40:15.188246 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:40:15.189114 (+ 868us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:40:15.189117 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:40:15.189712 (+ 595us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:40:15.189721 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:40:15.190194 (+ 473us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:40:15.190200 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:40:15.190338 (+ 138us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:40:15.190888 (+ 550us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 15:40:15.190907 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0809 15:40:15.192236 (+ 1329us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 15:40:15.193871 (+ 1635us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 15:40:15.193875 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 15:40:15.193877 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 15:40:15.194071 (+ 194us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 15:40:15.194340 (+ 269us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 15:40:15.194345 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 15:40:15.194347 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 15:40:15.194427 (+ 80us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 15:40:15.194804 (+ 377us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 15:40:15.194807 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 15:40:15.194890 (+ 83us) client_negotiation.cc:769] Sending connection context
0809 15:40:15.195054 (+ 164us) client_negotiation.cc:240] Negotiation successful
0809 15:40:15.195196 (+ 142us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":40}
I20260809 15:40:15.196162 27678 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.3.89.252" port: 35283 } has no permanent_uuid. Determining permanent_uuid...
I20260809 15:40:15.196272 3431 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskm314YR/build/debug/bin/kudu
/tmp/dist-test-taskm314YR/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786289732823074-3431-0/minicluster-data/master-2/wal
--fs_data_dirs=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786289732823074-3431-0/minicluster-data/master-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786289732823074-3431-0/minicluster-data/master-2/logs
--server_dump_info_path=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786289732823074-3431-0/minicluster-data/master-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.3.89.252:35283
--webserver_interface=127.3.89.252
--webserver_port=0
--keytab_file=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786289732823074-3431-0/krb5kdc/kudu_127.3.89.252.keytab
--principal=kudu/127.3.89.252
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.3.89.212:37439
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.3.89.254:33105,127.3.89.253:37127,127.3.89.252:35283
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=optional with env {KRB5CCNAME=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786289732823074-3431-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786289732823074-3431-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786289732823074-3431-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260809 15:40:15.196864 27682 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:40:15.196622 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.89.252:35283 (local address 127.0.0.1:50234)
0809 15:40:15.196732 (+ 110us) negotiation.cc:107] Waiting for socket to connect
0809 15:40:15.196808 (+ 76us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.89.252:35283: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":45}
W20260809 15:40:15.197690 27624 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.3.89.252:35283: connect: Connection refused (error 111)
I20260809 15:40:15.198640 27682 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:40:15.198377 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.89.252:35283 (local address 127.0.0.1:50248)
0809 15:40:15.198504 (+ 127us) negotiation.cc:107] Waiting for socket to connect
0809 15:40:15.198564 (+ 60us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.89.252:35283: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":49}
W20260809 15:40:15.198819 27678 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.3.89.252:35283: Network error: Client connection negotiation failed: client connection to 127.3.89.252:35283: connect: Connection refused (error 111)
I20260809 15:40:15.219691 27597 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.3.89.252" port: 35283 } attempt: 1
I20260809 15:40:15.220547 27602 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:40:15.220235 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.89.252:35283 (local address 127.0.0.1:50254)
0809 15:40:15.220394 (+ 159us) negotiation.cc:107] Waiting for socket to connect
0809 15:40:15.220466 (+ 72us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.89.252:35283: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":65}
I20260809 15:40:15.221303 27602 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:40:15.221084 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.89.252:35283 (local address 127.0.0.1:50258)
0809 15:40:15.221194 (+ 110us) negotiation.cc:107] Waiting for socket to connect
0809 15:40:15.221245 (+ 51us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.89.252:35283: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":45}
W20260809 15:40:15.221432 27597 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.3.89.252:35283: Network error: Client connection negotiation failed: client connection to 127.3.89.252:35283: connect: Connection refused (error 111)
I20260809 15:40:15.248099 27678 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.3.89.252" port: 35283 } attempt: 1
I20260809 15:40:15.249011 27682 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:40:15.248710 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.89.252:35283 (local address 127.0.0.1:50266)
0809 15:40:15.248868 (+ 158us) negotiation.cc:107] Waiting for socket to connect
0809 15:40:15.248931 (+ 63us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.89.252:35283: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":61}
I20260809 15:40:15.249732 27682 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:40:15.249529 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.89.252:35283 (local address 127.0.0.1:50272)
0809 15:40:15.249632 (+ 103us) negotiation.cc:107] Waiting for socket to connect
0809 15:40:15.249683 (+ 51us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.89.252:35283: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":39}
W20260809 15:40:15.249894 27678 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.3.89.252:35283: Network error: Client connection negotiation failed: client connection to 127.3.89.252:35283: connect: Connection refused (error 111)
I20260809 15:40:15.296587 27597 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.3.89.252" port: 35283 } attempt: 2
I20260809 15:40:15.297397 27602 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:40:15.297103 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.89.252:35283 (local address 127.0.0.1:50274)
0809 15:40:15.297251 (+ 148us) negotiation.cc:107] Waiting for socket to connect
0809 15:40:15.297316 (+ 65us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.89.252:35283: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":58}
I20260809 15:40:15.298207 27602 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:40:15.297994 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.89.252:35283 (local address 127.0.0.1:50282)
0809 15:40:15.298099 (+ 105us) negotiation.cc:107] Waiting for socket to connect
0809 15:40:15.298154 (+ 55us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.89.252:35283: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":43}
W20260809 15:40:15.298355 27597 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.3.89.252:35283: Network error: Client connection negotiation failed: client connection to 127.3.89.252:35283: connect: Connection refused (error 111)
W20260809 15:40:15.308485 27685 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260809 15:40:15.308781 27685 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 15:40:15.308843 27685 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 15:40:15.308919 27685 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 15:40:15.313066 27685 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 15:40:15.313133 27685 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 15:40:15.313181 27685 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 15:40:15.313216 27685 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 15:40:15.313251 27685 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 15:40:15.318035 27678 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.3.89.252" port: 35283 } attempt: 2
I20260809 15:40:15.318375 27685 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.3.89.212:37439
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786289732823074-3431-0/minicluster-data/master-2/data
--fs_wal_dir=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786289732823074-3431-0/minicluster-data/master-2/wal
--ipki_ca_key_size=768
--master_addresses=127.3.89.254:33105,127.3.89.253:37127,127.3.89.252:35283
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786289732823074-3431-0/krb5kdc/kudu_127.3.89.252.keytab
--principal=kudu/127.3.89.252
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.3.89.252:35283
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786289732823074-3431-0/minicluster-data/master-2/data/info.pb
--webserver_interface=127.3.89.252
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.27685
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786289732823074-3431-0/minicluster-data/master-2/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 09 Aug 2026 14:46:55 UTC on ccycloud.quasar-yswkmf.root.comops.site
build id 4576540
I20260809 15:40:15.318794 27682 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:40:15.318475 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.89.252:35283 (local address 127.0.0.1:50296)
0809 15:40:15.318680 (+ 205us) negotiation.cc:107] Waiting for socket to connect
0809 15:40:15.318728 (+ 48us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.89.252:35283: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":131}
I20260809 15:40:15.319468 27685 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 15:40:15.319505 27682 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:40:15.319296 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.89.252:35283 (local address 127.0.0.1:50308)
0809 15:40:15.319406 (+ 110us) negotiation.cc:107] Waiting for socket to connect
0809 15:40:15.319457 (+ 51us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.89.252:35283: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":45}
W20260809 15:40:15.319630 27678 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.3.89.252:35283: Network error: Client connection negotiation failed: client connection to 127.3.89.252:35283: connect: Connection refused (error 111)
I20260809 15:40:15.320463 27685 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 15:40:15.328752 27691 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 15:40:15.328734 27692 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 15:40:15.328841 27685 server_base.cc:1034] running on GCE node
W20260809 15:40:15.328996 27694 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 15:40:15.329526 27685 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 15:40:15.330554 27685 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 15:40:15.331746 27685 hybrid_clock.cc:648] HybridClock initialized: now 1786290015331719 us; error 36 us; skew 500 ppm
Aug 09 15:40:15 dist-test-slave-2w3w krb5kdc[27510](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786290015, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.89.252@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 15:40:15.334462 27685 init.cc:377] Logged in from keytab as kudu/127.3.89.252@KRBTEST.COM (short username kudu)
I20260809 15:40:15.335332 27685 webserver.cc:460] Webserver started at http://127.3.89.252:33603/ using document root <none> and password file <none>
I20260809 15:40:15.335819 27685 fs_manager.cc:359] Metadata directory not provided
I20260809 15:40:15.335875 27685 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 15:40:15.336045 27685 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 15:40:15.337970 27685 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskm314YR/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786289732823074-3431-0/minicluster-data/master-2/data/instance:
uuid: "9b24f2abb5ef4f7d86326819787081e1"
format_stamp: "Formatted at 2026-08-09 15:40:15 on dist-test-slave-2w3w"
server_key: "9e2c006468148c3979bc5cf970356afd"
server_key_iv: "0c336e2f2d9d3c44c853f4383d56f81e"
server_key_version: "encryptionkey@0"
I20260809 15:40:15.338478 27685 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskm314YR/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786289732823074-3431-0/minicluster-data/master-2/wal/instance:
uuid: "9b24f2abb5ef4f7d86326819787081e1"
format_stamp: "Formatted at 2026-08-09 15:40:15 on dist-test-slave-2w3w"
server_key: "9e2c006468148c3979bc5cf970356afd"
server_key_iv: "0c336e2f2d9d3c44c853f4383d56f81e"
server_key_version: "encryptionkey@0"
I20260809 15:40:15.342056 27685 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.006s sys 0.000s
I20260809 15:40:15.344375 27703 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 15:40:15.345633 27685 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260809 15:40:15.345769 27685 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskm314YR/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786289732823074-3431-0/minicluster-data/master-2/data,/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786289732823074-3431-0/minicluster-data/master-2/wal
uuid: "9b24f2abb5ef4f7d86326819787081e1"
format_stamp: "Formatted at 2026-08-09 15:40:15 on dist-test-slave-2w3w"
server_key: "9e2c006468148c3979bc5cf970356afd"
server_key_iv: "0c336e2f2d9d3c44c853f4383d56f81e"
server_key_version: "encryptionkey@0"
I20260809 15:40:15.345902 27685 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskm314YR/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786289732823074-3431-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-taskm314YR/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786289732823074-3431-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-taskm314YR/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786289732823074-3431-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 15:40:15.362905 27685 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 15:40:15.365581 27685 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 15:40:15.365726 27685 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 15:40:15.372527 27685 rpc_server.cc:307] RPC server started. Bound to: 127.3.89.252:35283
I20260809 15:40:15.372802 27760 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.3.89.252:35283 every 8 connection(s)
I20260809 15:40:15.373462 27685 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskm314YR/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786289732823074-3431-0/minicluster-data/master-2/data/info.pb
I20260809 15:40:15.376121 27761 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 15:40:15.379638 27761 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.3.89.254" port: 33105 } has no permanent_uuid. Determining permanent_uuid...
I20260809 15:40:15.383769 3431 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskm314YR/build/debug/bin/kudu as pid 27685
I20260809 15:40:15.383900 3431 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskm314YR/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786289732823074-3431-0/minicluster-data/master-2/wal/instance
I20260809 15:40:15.384184 3431 external_mini_cluster.cc:1428] Setting key b4062a4e423ea613539676d35a1f40d7
Aug 09 15:40:15 dist-test-slave-2w3w krb5kdc[27510](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786290015, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.89.252@KRBTEST.COM for kudu/127.3.89.254@KRBTEST.COM
Aug 09 15:40:15 dist-test-slave-2w3w krb5kdc[27510](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786290014, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.3.89.252@KRBTEST.COM
I20260809 15:40:15.394017 27600 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:40:15.381415 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45258 (local address 127.3.89.254:33105)
0809 15:40:15.381554 (+ 139us) server_negotiation.cc:206] Beginning negotiation
0809 15:40:15.381558 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 15:40:15.382314 (+ 756us) server_negotiation.cc:407] Connection header received
0809 15:40:15.383279 (+ 965us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 15:40:15.383290 (+ 11us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 15:40:15.383357 (+ 67us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 15:40:15.383455 (+ 98us) server_negotiation.cc:226] Negotiated authn=SASL
0809 15:40:15.384825 (+ 1370us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:40:15.385606 (+ 781us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:40:15.387165 (+ 1559us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:40:15.387545 (+ 380us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:40:15.387672 (+ 127us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:40:15.390165 (+ 2493us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 15:40:15.390168 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 15:40:15.390170 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 15:40:15.390195 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0809 15:40:15.391816 (+ 1621us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 15:40:15.392399 (+ 583us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 15:40:15.392403 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 15:40:15.392407 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 15:40:15.392524 (+ 117us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 15:40:15.392928 (+ 404us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 15:40:15.392934 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 15:40:15.392938 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 15:40:15.393198 (+ 260us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 15:40:15.393313 (+ 115us) server_negotiation.cc:1035] Waiting for connection context
0809 15:40:15.393772 (+ 459us) server_negotiation.cc:299] Negotiation successful
0809 15:40:15.393896 (+ 124us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":55}
I20260809 15:40:15.395040 27763 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:40:15.381719 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.89.254:33105 (local address 127.0.0.1:45258)
0809 15:40:15.382145 (+ 426us) negotiation.cc:107] Waiting for socket to connect
0809 15:40:15.382186 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0809 15:40:15.383067 (+ 881us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 15:40:15.383626 (+ 559us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 15:40:15.383633 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 15:40:15.384066 (+ 433us) client_negotiation.cc:189] Negotiated authn=SASL
0809 15:40:15.384646 (+ 580us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:40:15.384670 (+ 24us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:40:15.385738 (+ 1068us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:40:15.385742 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:40:15.387054 (+ 1312us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:40:15.387063 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:40:15.387721 (+ 658us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:40:15.387726 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:40:15.387924 (+ 198us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:40:15.388490 (+ 566us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 15:40:15.388510 (+ 20us) client_negotiation.cc:656] calling sasl_client_start()
0809 15:40:15.390014 (+ 1504us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 15:40:15.391920 (+ 1906us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 15:40:15.391928 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 15:40:15.391935 (+ 7us) client_negotiation.cc:760] calling sasl_client_step()
0809 15:40:15.392272 (+ 337us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 15:40:15.392629 (+ 357us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 15:40:15.392631 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 15:40:15.392632 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0809 15:40:15.392823 (+ 191us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 15:40:15.393329 (+ 506us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 15:40:15.393333 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 15:40:15.393537 (+ 204us) client_negotiation.cc:769] Sending connection context
0809 15:40:15.393685 (+ 148us) client_negotiation.cc:240] Negotiation successful
0809 15:40:15.393989 (+ 304us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":247,"thread_start_us":101,"threads_started":1}
I20260809 15:40:15.395911 27765 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:40:15.384873 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50312 (local address 127.3.89.252:35283)
0809 15:40:15.385227 (+ 354us) server_negotiation.cc:206] Beginning negotiation
0809 15:40:15.385237 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0809 15:40:15.385260 (+ 23us) server_negotiation.cc:407] Connection header received
0809 15:40:15.385306 (+ 46us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 15:40:15.385311 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 15:40:15.385496 (+ 185us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 15:40:15.385664 (+ 168us) server_negotiation.cc:226] Negotiated authn=SASL
0809 15:40:15.386616 (+ 952us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:40:15.387596 (+ 980us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:40:15.388687 (+ 1091us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:40:15.389006 (+ 319us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:40:15.389084 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:40:15.391798 (+ 2714us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 15:40:15.391806 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 15:40:15.391817 (+ 11us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 15:40:15.391843 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0809 15:40:15.393692 (+ 1849us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 15:40:15.394194 (+ 502us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 15:40:15.394200 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 15:40:15.394202 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 15:40:15.394289 (+ 87us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 15:40:15.394579 (+ 290us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 15:40:15.394582 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 15:40:15.394584 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 15:40:15.394858 (+ 274us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 15:40:15.395092 (+ 234us) server_negotiation.cc:1035] Waiting for connection context
0809 15:40:15.395631 (+ 539us) server_negotiation.cc:299] Negotiation successful
0809 15:40:15.395780 (+ 149us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":269,"thread_start_us":90,"threads_started":1}
I20260809 15:40:15.396353 27761 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.3.89.253" port: 37127 } has no permanent_uuid. Determining permanent_uuid...
I20260809 15:40:15.397514 27597 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.3.89.252" port: 35283 } attempt: 3
Aug 09 15:40:15 dist-test-slave-2w3w krb5kdc[27510](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786290015, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.89.252@KRBTEST.COM for kudu/127.3.89.253@KRBTEST.COM
Aug 09 15:40:15 dist-test-slave-2w3w krb5kdc[27510](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786290014, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.89.254@KRBTEST.COM for kudu/127.3.89.252@KRBTEST.COM
Aug 09 15:40:15 dist-test-slave-2w3w krb5kdc[27510](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786290014, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.3.89.254@KRBTEST.COM
I20260809 15:40:15.405699 27763 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:40:15.396834 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.89.253:37127 (local address 127.0.0.1:47344)
0809 15:40:15.396925 (+ 91us) negotiation.cc:107] Waiting for socket to connect
0809 15:40:15.396938 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0809 15:40:15.397009 (+ 71us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 15:40:15.397314 (+ 305us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 15:40:15.397316 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 15:40:15.397437 (+ 121us) client_negotiation.cc:189] Negotiated authn=SASL
0809 15:40:15.397523 (+ 86us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:40:15.397527 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:40:15.398498 (+ 971us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:40:15.398501 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:40:15.399316 (+ 815us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:40:15.399325 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:40:15.400003 (+ 678us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:40:15.400007 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:40:15.400110 (+ 103us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:40:15.400593 (+ 483us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 15:40:15.400607 (+ 14us) client_negotiation.cc:656] calling sasl_client_start()
0809 15:40:15.401960 (+ 1353us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 15:40:15.404023 (+ 2063us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 15:40:15.404028 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 15:40:15.404030 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 15:40:15.404299 (+ 269us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 15:40:15.404577 (+ 278us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 15:40:15.404582 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 15:40:15.404584 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 15:40:15.404692 (+ 108us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 15:40:15.405107 (+ 415us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 15:40:15.405112 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 15:40:15.405201 (+ 89us) client_negotiation.cc:769] Sending connection context
0809 15:40:15.405316 (+ 115us) client_negotiation.cc:240] Negotiation successful
0809 15:40:15.405433 (+ 117us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":24}
I20260809 15:40:15.406178 27683 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:40:15.396945 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47344 (local address 127.3.89.253:37127)
0809 15:40:15.397079 (+ 134us) server_negotiation.cc:206] Beginning negotiation
0809 15:40:15.397083 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 15:40:15.397098 (+ 15us) server_negotiation.cc:407] Connection header received
0809 15:40:15.397144 (+ 46us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 15:40:15.397146 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 15:40:15.397193 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 15:40:15.397284 (+ 91us) server_negotiation.cc:226] Negotiated authn=SASL
0809 15:40:15.397647 (+ 363us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:40:15.398374 (+ 727us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:40:15.399442 (+ 1068us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:40:15.399774 (+ 332us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:40:15.399876 (+ 102us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:40:15.402114 (+ 2238us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 15:40:15.402120 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 15:40:15.402124 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 15:40:15.402154 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0809 15:40:15.403910 (+ 1756us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 15:40:15.404388 (+ 478us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 15:40:15.404393 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 15:40:15.404395 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 15:40:15.404483 (+ 88us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 15:40:15.404814 (+ 331us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 15:40:15.404817 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 15:40:15.404819 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 15:40:15.404999 (+ 180us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 15:40:15.405105 (+ 106us) server_negotiation.cc:1035] Waiting for connection context
0809 15:40:15.405368 (+ 263us) server_negotiation.cc:299] Negotiation successful
0809 15:40:15.405990 (+ 622us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
I20260809 15:40:15.406824 27761 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.3.89.252" port: 35283 } has no permanent_uuid. Determining permanent_uuid...
I20260809 15:40:15.406903 27602 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:40:15.397900 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.89.252:35283 (local address 127.0.0.1:50322)
0809 15:40:15.398113 (+ 213us) negotiation.cc:107] Waiting for socket to connect
0809 15:40:15.398125 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0809 15:40:15.398192 (+ 67us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 15:40:15.398648 (+ 456us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 15:40:15.398650 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 15:40:15.398834 (+ 184us) client_negotiation.cc:189] Negotiated authn=SASL
0809 15:40:15.398950 (+ 116us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:40:15.398986 (+ 36us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:40:15.399776 (+ 790us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:40:15.399779 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:40:15.400354 (+ 575us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:40:15.400365 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:40:15.400981 (+ 616us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:40:15.400985 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:40:15.401069 (+ 84us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:40:15.401539 (+ 470us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 15:40:15.401553 (+ 14us) client_negotiation.cc:656] calling sasl_client_start()
0809 15:40:15.403043 (+ 1490us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 15:40:15.405129 (+ 2086us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 15:40:15.405135 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 15:40:15.405140 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0809 15:40:15.405427 (+ 287us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 15:40:15.405790 (+ 363us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 15:40:15.405795 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 15:40:15.405799 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0809 15:40:15.405894 (+ 95us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 15:40:15.406388 (+ 494us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 15:40:15.406393 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 15:40:15.406496 (+ 103us) client_negotiation.cc:769] Sending connection context
0809 15:40:15.406583 (+ 87us) client_negotiation.cc:240] Negotiation successful
0809 15:40:15.406717 (+ 134us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":150}
I20260809 15:40:15.407579 27765 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:40:15.398330 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50322 (local address 127.3.89.252:35283)
0809 15:40:15.398443 (+ 113us) server_negotiation.cc:206] Beginning negotiation
0809 15:40:15.398449 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 15:40:15.398462 (+ 13us) server_negotiation.cc:407] Connection header received
0809 15:40:15.398495 (+ 33us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 15:40:15.398498 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 15:40:15.398535 (+ 37us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 15:40:15.398608 (+ 73us) server_negotiation.cc:226] Negotiated authn=SASL
0809 15:40:15.399076 (+ 468us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:40:15.399670 (+ 594us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:40:15.400494 (+ 824us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:40:15.400837 (+ 343us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:40:15.401267 (+ 430us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:40:15.403185 (+ 1918us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 15:40:15.403191 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 15:40:15.403197 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 15:40:15.403229 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0809 15:40:15.404990 (+ 1761us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 15:40:15.405548 (+ 558us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 15:40:15.405551 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 15:40:15.405553 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 15:40:15.405671 (+ 118us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 15:40:15.406032 (+ 361us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 15:40:15.406035 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 15:40:15.406037 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 15:40:15.406259 (+ 222us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 15:40:15.407212 (+ 953us) server_negotiation.cc:1035] Waiting for connection context
0809 15:40:15.407306 (+ 94us) server_negotiation.cc:299] Negotiation successful
0809 15:40:15.407412 (+ 106us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":41}
I20260809 15:40:15.409055 27600 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:40:15.398415 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45266 (local address 127.3.89.254:33105)
0809 15:40:15.398515 (+ 100us) server_negotiation.cc:206] Beginning negotiation
0809 15:40:15.398519 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 15:40:15.398646 (+ 127us) server_negotiation.cc:407] Connection header received
0809 15:40:15.398824 (+ 178us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 15:40:15.398828 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 15:40:15.398868 (+ 40us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 15:40:15.398944 (+ 76us) server_negotiation.cc:226] Negotiated authn=SASL
0809 15:40:15.399644 (+ 700us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:40:15.400317 (+ 673us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:40:15.401416 (+ 1099us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:40:15.401767 (+ 351us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:40:15.401874 (+ 107us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:40:15.405219 (+ 3345us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 15:40:15.405223 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 15:40:15.405226 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 15:40:15.405249 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0809 15:40:15.407088 (+ 1839us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 15:40:15.407593 (+ 505us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 15:40:15.407598 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 15:40:15.407600 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 15:40:15.407689 (+ 89us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 15:40:15.408032 (+ 343us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 15:40:15.408036 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 15:40:15.408037 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 15:40:15.408266 (+ 229us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 15:40:15.408446 (+ 180us) server_negotiation.cc:1035] Waiting for connection context
0809 15:40:15.408764 (+ 318us) server_negotiation.cc:299] Negotiation successful
0809 15:40:15.408878 (+ 114us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":36}
I20260809 15:40:15.410785 27678 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.3.89.252" port: 35283 } attempt: 3
I20260809 15:40:15.410933 27597 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 76ee3afa6e0e4189a630938d1e316744: Bootstrap starting.
Aug 09 15:40:15 dist-test-slave-2w3w krb5kdc[27510](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786290015, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.89.252@KRBTEST.COM for kudu/127.3.89.252@KRBTEST.COM
I20260809 15:40:15.414006 27597 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 76ee3afa6e0e4189a630938d1e316744: Neither blocks nor log segments found. Creating new log.
I20260809 15:40:15.414898 27597 log.cc:824] T 00000000000000000000000000000000 P 76ee3afa6e0e4189a630938d1e316744: Log is configured to *not* fsync() on all Append() calls
Aug 09 15:40:15 dist-test-slave-2w3w krb5kdc[27510](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786290015, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.89.253@KRBTEST.COM for kudu/127.3.89.252@KRBTEST.COM
I20260809 15:40:15.416597 27763 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:40:15.407293 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.89.252:35283 (local address 127.0.0.1:50336)
0809 15:40:15.407581 (+ 288us) negotiation.cc:107] Waiting for socket to connect
0809 15:40:15.407595 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0809 15:40:15.407782 (+ 187us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 15:40:15.408007 (+ 225us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 15:40:15.408009 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 15:40:15.408190 (+ 181us) client_negotiation.cc:189] Negotiated authn=SASL
0809 15:40:15.408290 (+ 100us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:40:15.408299 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:40:15.409858 (+ 1559us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:40:15.409861 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:40:15.410340 (+ 479us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:40:15.410348 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:40:15.410904 (+ 556us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:40:15.410907 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:40:15.411055 (+ 148us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:40:15.411600 (+ 545us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 15:40:15.411616 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0809 15:40:15.412990 (+ 1374us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 15:40:15.414930 (+ 1940us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 15:40:15.414933 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 15:40:15.414935 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 15:40:15.415241 (+ 306us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 15:40:15.415539 (+ 298us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 15:40:15.415542 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 15:40:15.415544 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 15:40:15.415628 (+ 84us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 15:40:15.416112 (+ 484us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 15:40:15.416114 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 15:40:15.416194 (+ 80us) client_negotiation.cc:769] Sending connection context
0809 15:40:15.416296 (+ 102us) client_negotiation.cc:240] Negotiation successful
0809 15:40:15.416426 (+ 130us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":230,"mutex_wait_us":28,"spinlock_wait_cycles":1920}
I20260809 15:40:15.417075 27767 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:40:15.407351 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50336 (local address 127.3.89.252:35283)
0809 15:40:15.407596 (+ 245us) server_negotiation.cc:206] Beginning negotiation
0809 15:40:15.407605 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0809 15:40:15.407641 (+ 36us) server_negotiation.cc:407] Connection header received
0809 15:40:15.407869 (+ 228us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 15:40:15.407872 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 15:40:15.407906 (+ 34us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 15:40:15.407999 (+ 93us) server_negotiation.cc:226] Negotiated authn=SASL
0809 15:40:15.408401 (+ 402us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:40:15.409744 (+ 1343us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:40:15.410458 (+ 714us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:40:15.410779 (+ 321us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:40:15.410887 (+ 108us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:40:15.413130 (+ 2243us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 15:40:15.413133 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 15:40:15.413136 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 15:40:15.413155 (+ 19us) server_negotiation.cc:922] calling sasl_server_start()
0809 15:40:15.414835 (+ 1680us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 15:40:15.415347 (+ 512us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 15:40:15.415349 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 15:40:15.415351 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 15:40:15.415432 (+ 81us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 15:40:15.415714 (+ 282us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 15:40:15.415721 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 15:40:15.415723 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 15:40:15.415953 (+ 230us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 15:40:15.416071 (+ 118us) server_negotiation.cc:1035] Waiting for connection context
0809 15:40:15.416777 (+ 706us) server_negotiation.cc:299] Negotiation successful
0809 15:40:15.416884 (+ 107us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":82,"server-negotiator.queue_time_us":197,"thread_start_us":122,"threads_started":1}
I20260809 15:40:15.418140 27597 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 76ee3afa6e0e4189a630938d1e316744: No bootstrap required, opened a new log
I20260809 15:40:15.420013 27761 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 9b24f2abb5ef4f7d86326819787081e1: Bootstrap starting.
I20260809 15:40:15.420935 27682 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:40:15.411200 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.89.252:35283 (local address 127.0.0.1:50340)
0809 15:40:15.411557 (+ 357us) negotiation.cc:107] Waiting for socket to connect
0809 15:40:15.411573 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0809 15:40:15.411645 (+ 72us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 15:40:15.412008 (+ 363us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 15:40:15.412012 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 15:40:15.412206 (+ 194us) client_negotiation.cc:189] Negotiated authn=SASL
0809 15:40:15.412310 (+ 104us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:40:15.412329 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:40:15.413359 (+ 1030us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:40:15.413363 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:40:15.414147 (+ 784us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:40:15.414159 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:40:15.414786 (+ 627us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:40:15.414790 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:40:15.414881 (+ 91us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:40:15.415364 (+ 483us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 15:40:15.415379 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0809 15:40:15.416915 (+ 1536us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 15:40:15.419342 (+ 2427us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 15:40:15.419345 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 15:40:15.419347 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 15:40:15.419597 (+ 250us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 15:40:15.419905 (+ 308us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 15:40:15.419908 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 15:40:15.419910 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 15:40:15.420028 (+ 118us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 15:40:15.420452 (+ 424us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 15:40:15.420455 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 15:40:15.420554 (+ 99us) client_negotiation.cc:769] Sending connection context
0809 15:40:15.420666 (+ 112us) client_negotiation.cc:240] Negotiation successful
0809 15:40:15.420793 (+ 127us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":39}
I20260809 15:40:15.420986 27765 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:40:15.411479 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50340 (local address 127.3.89.252:35283)
0809 15:40:15.411592 (+ 113us) server_negotiation.cc:206] Beginning negotiation
0809 15:40:15.411597 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 15:40:15.411610 (+ 13us) server_negotiation.cc:407] Connection header received
0809 15:40:15.411742 (+ 132us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 15:40:15.411745 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 15:40:15.411788 (+ 43us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 15:40:15.411872 (+ 84us) server_negotiation.cc:226] Negotiated authn=SASL
0809 15:40:15.412464 (+ 592us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:40:15.413243 (+ 779us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:40:15.414285 (+ 1042us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:40:15.414637 (+ 352us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:40:15.414771 (+ 134us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:40:15.417052 (+ 2281us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 15:40:15.417055 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 15:40:15.417058 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 15:40:15.417082 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0809 15:40:15.419222 (+ 2140us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 15:40:15.419703 (+ 481us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 15:40:15.419707 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 15:40:15.419709 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 15:40:15.419794 (+ 85us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 15:40:15.420133 (+ 339us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 15:40:15.420136 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 15:40:15.420138 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 15:40:15.420330 (+ 192us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 15:40:15.420437 (+ 107us) server_negotiation.cc:1035] Waiting for connection context
0809 15:40:15.420732 (+ 295us) server_negotiation.cc:299] Negotiation successful
0809 15:40:15.420850 (+ 118us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":42}
I20260809 15:40:15.422322 27597 raft_consensus.cc:348] T 00000000000000000000000000000000 P 76ee3afa6e0e4189a630938d1e316744 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "76ee3afa6e0e4189a630938d1e316744" member_type: VOTER last_known_addr { host: "127.3.89.254" port: 33105 } } peers { permanent_uuid: "1d43cc5aa2554fdf851c3663c9717f3c" member_type: VOTER last_known_addr { host: "127.3.89.253" port: 37127 } } peers { permanent_uuid: "9b24f2abb5ef4f7d86326819787081e1" member_type: VOTER last_known_addr { host: "127.3.89.252" port: 35283 } }
I20260809 15:40:15.422520 27761 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 9b24f2abb5ef4f7d86326819787081e1: Neither blocks nor log segments found. Creating new log.
I20260809 15:40:15.423398 27761 log.cc:824] T 00000000000000000000000000000000 P 9b24f2abb5ef4f7d86326819787081e1: Log is configured to *not* fsync() on all Append() calls
I20260809 15:40:15.423895 27678 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1d43cc5aa2554fdf851c3663c9717f3c: Bootstrap starting.
I20260809 15:40:15.425179 27761 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 9b24f2abb5ef4f7d86326819787081e1: No bootstrap required, opened a new log
I20260809 15:40:15.426349 27678 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 1d43cc5aa2554fdf851c3663c9717f3c: Neither blocks nor log segments found. Creating new log.
I20260809 15:40:15.426770 27597 raft_consensus.cc:729] T 00000000000000000000000000000000 P 76ee3afa6e0e4189a630938d1e316744 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 76ee3afa6e0e4189a630938d1e316744, State: Initialized, Role: FOLLOWER
I20260809 15:40:15.427227 27678 log.cc:824] T 00000000000000000000000000000000 P 1d43cc5aa2554fdf851c3663c9717f3c: Log is configured to *not* fsync() on all Append() calls
Aug 09 15:40:15 dist-test-slave-2w3w krb5kdc[27510](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786290014, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.3.89.253@KRBTEST.COM
I20260809 15:40:15.428647 27761 raft_consensus.cc:348] T 00000000000000000000000000000000 P 9b24f2abb5ef4f7d86326819787081e1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "76ee3afa6e0e4189a630938d1e316744" member_type: VOTER last_known_addr { host: "127.3.89.254" port: 33105 } } peers { permanent_uuid: "1d43cc5aa2554fdf851c3663c9717f3c" member_type: VOTER last_known_addr { host: "127.3.89.253" port: 37127 } } peers { permanent_uuid: "9b24f2abb5ef4f7d86326819787081e1" member_type: VOTER last_known_addr { host: "127.3.89.252" port: 35283 } }
I20260809 15:40:15.428910 27761 raft_consensus.cc:729] T 00000000000000000000000000000000 P 9b24f2abb5ef4f7d86326819787081e1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9b24f2abb5ef4f7d86326819787081e1, State: Initialized, Role: FOLLOWER
I20260809 15:40:15.429210 27761 consensus_queue.cc:260] T 00000000000000000000000000000000 P 9b24f2abb5ef4f7d86326819787081e1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "76ee3afa6e0e4189a630938d1e316744" member_type: VOTER last_known_addr { host: "127.3.89.254" port: 33105 } } peers { permanent_uuid: "1d43cc5aa2554fdf851c3663c9717f3c" member_type: VOTER last_known_addr { host: "127.3.89.253" port: 37127 } } peers { permanent_uuid: "9b24f2abb5ef4f7d86326819787081e1" member_type: VOTER last_known_addr { host: "127.3.89.252" port: 35283 } }
I20260809 15:40:15.429344 27678 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1d43cc5aa2554fdf851c3663c9717f3c: No bootstrap required, opened a new log
I20260809 15:40:15.429672 27771 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9b24f2abb5ef4f7d86326819787081e1 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "76ee3afa6e0e4189a630938d1e316744" member_type: VOTER last_known_addr { host: "127.3.89.254" port: 33105 } } peers { permanent_uuid: "1d43cc5aa2554fdf851c3663c9717f3c" member_type: VOTER last_known_addr { host: "127.3.89.253" port: 37127 } } peers { permanent_uuid: "9b24f2abb5ef4f7d86326819787081e1" member_type: VOTER last_known_addr { host: "127.3.89.252" port: 35283 } } }
I20260809 15:40:15.429874 27771 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9b24f2abb5ef4f7d86326819787081e1 [sys.catalog]: This master's current role is: FOLLOWER
I20260809 15:40:15.430301 27597 consensus_queue.cc:260] T 00000000000000000000000000000000 P 76ee3afa6e0e4189a630938d1e316744 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "76ee3afa6e0e4189a630938d1e316744" member_type: VOTER last_known_addr { host: "127.3.89.254" port: 33105 } } peers { permanent_uuid: "1d43cc5aa2554fdf851c3663c9717f3c" member_type: VOTER last_known_addr { host: "127.3.89.253" port: 37127 } } peers { permanent_uuid: "9b24f2abb5ef4f7d86326819787081e1" member_type: VOTER last_known_addr { host: "127.3.89.252" port: 35283 } }
I20260809 15:40:15.430631 27761 sys_catalog.cc:564] T 00000000000000000000000000000000 P 9b24f2abb5ef4f7d86326819787081e1 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 15:40:15.431598 27597 sys_catalog.cc:564] T 00000000000000000000000000000000 P 76ee3afa6e0e4189a630938d1e316744 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 15:40:15.433491 27678 raft_consensus.cc:348] T 00000000000000000000000000000000 P 1d43cc5aa2554fdf851c3663c9717f3c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "76ee3afa6e0e4189a630938d1e316744" member_type: VOTER last_known_addr { host: "127.3.89.254" port: 33105 } } peers { permanent_uuid: "1d43cc5aa2554fdf851c3663c9717f3c" member_type: VOTER last_known_addr { host: "127.3.89.253" port: 37127 } } peers { permanent_uuid: "9b24f2abb5ef4f7d86326819787081e1" member_type: VOTER last_known_addr { host: "127.3.89.252" port: 35283 } }
I20260809 15:40:15.433717 27678 raft_consensus.cc:729] T 00000000000000000000000000000000 P 1d43cc5aa2554fdf851c3663c9717f3c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1d43cc5aa2554fdf851c3663c9717f3c, State: Initialized, Role: FOLLOWER
I20260809 15:40:15.434036 27678 consensus_queue.cc:260] T 00000000000000000000000000000000 P 1d43cc5aa2554fdf851c3663c9717f3c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "76ee3afa6e0e4189a630938d1e316744" member_type: VOTER last_known_addr { host: "127.3.89.254" port: 33105 } } peers { permanent_uuid: "1d43cc5aa2554fdf851c3663c9717f3c" member_type: VOTER last_known_addr { host: "127.3.89.253" port: 37127 } } peers { permanent_uuid: "9b24f2abb5ef4f7d86326819787081e1" member_type: VOTER last_known_addr { host: "127.3.89.252" port: 35283 } }
I20260809 15:40:15.435274 27678 sys_catalog.cc:564] T 00000000000000000000000000000000 P 1d43cc5aa2554fdf851c3663c9717f3c [sys.catalog]: configured and running, proceeding with master startup.
I20260809 15:40:15.435585 27683 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:40:15.416429 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47354 (local address 127.3.89.253:37127)
0809 15:40:15.416548 (+ 119us) server_negotiation.cc:206] Beginning negotiation
0809 15:40:15.416553 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 15:40:15.422764 (+ 6211us) server_negotiation.cc:407] Connection header received
0809 15:40:15.422954 (+ 190us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 15:40:15.423004 (+ 50us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 15:40:15.423053 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 15:40:15.423102 (+ 49us) server_negotiation.cc:226] Negotiated authn=SASL
0809 15:40:15.424047 (+ 945us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:40:15.424755 (+ 708us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:40:15.425854 (+ 1099us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:40:15.426187 (+ 333us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:40:15.426275 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:40:15.429281 (+ 3006us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 15:40:15.429285 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 15:40:15.429287 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 15:40:15.429322 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0809 15:40:15.431064 (+ 1742us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 15:40:15.433591 (+ 2527us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 15:40:15.433595 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 15:40:15.433596 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 15:40:15.433681 (+ 85us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 15:40:15.434522 (+ 841us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 15:40:15.434525 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 15:40:15.434527 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 15:40:15.434764 (+ 237us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 15:40:15.434907 (+ 143us) server_negotiation.cc:1035] Waiting for connection context
0809 15:40:15.435304 (+ 397us) server_negotiation.cc:299] Negotiation successful
0809 15:40:15.435439 (+ 135us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
W20260809 15:40:15.436046 27789 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 9b24f2abb5ef4f7d86326819787081e1: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260809 15:40:15.436112 27789 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260809 15:40:15.434743 27685 master_runner.cc:186] Error getting master registration for 127.3.89.254:33105: OK, instance_id { permanent_uuid: "76ee3afa6e0e4189a630938d1e316744" instance_seqno: 1786290014930112 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260809 15:40:15.437914 27685 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260809 15:40:15.438468 27772 sys_catalog.cc:455] T 00000000000000000000000000000000 P 76ee3afa6e0e4189a630938d1e316744 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "76ee3afa6e0e4189a630938d1e316744" member_type: VOTER last_known_addr { host: "127.3.89.254" port: 33105 } } peers { permanent_uuid: "1d43cc5aa2554fdf851c3663c9717f3c" member_type: VOTER last_known_addr { host: "127.3.89.253" port: 37127 } } peers { permanent_uuid: "9b24f2abb5ef4f7d86326819787081e1" member_type: VOTER last_known_addr { host: "127.3.89.252" port: 35283 } } }
I20260809 15:40:15.438594 27772 sys_catalog.cc:458] T 00000000000000000000000000000000 P 76ee3afa6e0e4189a630938d1e316744 [sys.catalog]: This master's current role is: FOLLOWER
I20260809 15:40:15.439254 3431 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
W20260809 15:40:15.441952 27810 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 76ee3afa6e0e4189a630938d1e316744: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260809 15:40:15.442025 27810 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260809 15:40:15.443672 27605 master_runner.cc:263] No leader master found from master 1d43cc5aa2554fdf851c3663c9717f3c
I20260809 15:40:15.443738 27605 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260809 15:40:15.443665 27569 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "00000000000000000000000000000000"
dest_uuid: "76ee3afa6e0e4189a630938d1e316744"
from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:45266
I20260809 15:40:15.443922 27569 raft_consensus.cc:482] T 00000000000000000000000000000000 P 76ee3afa6e0e4189a630938d1e316744 [term 0 FOLLOWER]: Starting forced leader election (received explicit request)
I20260809 15:40:15.444043 27569 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 76ee3afa6e0e4189a630938d1e316744 [term 0 FOLLOWER]: Advancing to term 1
W20260809 15:40:15.444649 27804 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 1d43cc5aa2554fdf851c3663c9717f3c: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260809 15:40:15.444725 27804 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260809 15:40:15.445502 27569 raft_consensus.cc:504] T 00000000000000000000000000000000 P 76ee3afa6e0e4189a630938d1e316744 [term 1 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "76ee3afa6e0e4189a630938d1e316744" member_type: VOTER last_known_addr { host: "127.3.89.254" port: 33105 } } peers { permanent_uuid: "1d43cc5aa2554fdf851c3663c9717f3c" member_type: VOTER last_known_addr { host: "127.3.89.253" port: 37127 } } peers { permanent_uuid: "9b24f2abb5ef4f7d86326819787081e1" member_type: VOTER last_known_addr { host: "127.3.89.252" port: 35283 } }
I20260809 15:40:15.446854 27731 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "76ee3afa6e0e4189a630938d1e316744" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: true dest_uuid: "9b24f2abb5ef4f7d86326819787081e1"
I20260809 15:40:15.447105 27731 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 9b24f2abb5ef4f7d86326819787081e1 [term 0 FOLLOWER]: Advancing to term 1
I20260809 15:40:15.447139 27523 master_runner.cc:263] No leader master found from master 76ee3afa6e0e4189a630938d1e316744
I20260809 15:40:15.447192 27523 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260809 15:40:15.447240 27569 leader_election.cc:288] T 00000000000000000000000000000000 P 76ee3afa6e0e4189a630938d1e316744 [CANDIDATE]: Term 1 election: Requested vote from peers 1d43cc5aa2554fdf851c3663c9717f3c (127.3.89.253:37127), 9b24f2abb5ef4f7d86326819787081e1 (127.3.89.252:35283)
I20260809 15:40:15.448431 27731 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 9b24f2abb5ef4f7d86326819787081e1 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 76ee3afa6e0e4189a630938d1e316744 in term 1.
I20260809 15:40:15.448808 27649 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "76ee3afa6e0e4189a630938d1e316744" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: true dest_uuid: "1d43cc5aa2554fdf851c3663c9717f3c"
I20260809 15:40:15.449010 27649 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 1d43cc5aa2554fdf851c3663c9717f3c [term 0 FOLLOWER]: Advancing to term 1
I20260809 15:40:15.450353 27649 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 1d43cc5aa2554fdf851c3663c9717f3c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 76ee3afa6e0e4189a630938d1e316744 in term 1.
I20260809 15:40:15.451041 27544 leader_election.cc:302] T 00000000000000000000000000000000 P 76ee3afa6e0e4189a630938d1e316744 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 76ee3afa6e0e4189a630938d1e316744, 9b24f2abb5ef4f7d86326819787081e1; no voters:
I20260809 15:40:15.451614 27772 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 76ee3afa6e0e4189a630938d1e316744 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 15:40:15.451903 27772 raft_consensus.cc:686] T 00000000000000000000000000000000 P 76ee3afa6e0e4189a630938d1e316744 [term 1 LEADER]: Becoming Leader. State: Replica: 76ee3afa6e0e4189a630938d1e316744, State: Running, Role: LEADER
I20260809 15:40:15.452283 27772 consensus_queue.cc:237] T 00000000000000000000000000000000 P 76ee3afa6e0e4189a630938d1e316744 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "76ee3afa6e0e4189a630938d1e316744" member_type: VOTER last_known_addr { host: "127.3.89.254" port: 33105 } } peers { permanent_uuid: "1d43cc5aa2554fdf851c3663c9717f3c" member_type: VOTER last_known_addr { host: "127.3.89.253" port: 37127 } } peers { permanent_uuid: "9b24f2abb5ef4f7d86326819787081e1" member_type: VOTER last_known_addr { host: "127.3.89.252" port: 35283 } }
I20260809 15:40:15.454192 27772 sys_catalog.cc:455] T 00000000000000000000000000000000 P 76ee3afa6e0e4189a630938d1e316744 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 76ee3afa6e0e4189a630938d1e316744. Latest consensus state: current_term: 1 leader_uuid: "76ee3afa6e0e4189a630938d1e316744" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "76ee3afa6e0e4189a630938d1e316744" member_type: VOTER last_known_addr { host: "127.3.89.254" port: 33105 } } peers { permanent_uuid: "1d43cc5aa2554fdf851c3663c9717f3c" member_type: VOTER last_known_addr { host: "127.3.89.253" port: 37127 } } peers { permanent_uuid: "9b24f2abb5ef4f7d86326819787081e1" member_type: VOTER last_known_addr { host: "127.3.89.252" port: 35283 } } }
I20260809 15:40:15.454385 27772 sys_catalog.cc:458] T 00000000000000000000000000000000 P 76ee3afa6e0e4189a630938d1e316744 [sys.catalog]: This master's current role is: LEADER
I20260809 15:40:15.454855 27819 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 15:40:15.455060 27790 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1d43cc5aa2554fdf851c3663c9717f3c [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "76ee3afa6e0e4189a630938d1e316744" member_type: VOTER last_known_addr { host: "127.3.89.254" port: 33105 } } peers { permanent_uuid: "1d43cc5aa2554fdf851c3663c9717f3c" member_type: VOTER last_known_addr { host: "127.3.89.253" port: 37127 } } peers { permanent_uuid: "9b24f2abb5ef4f7d86326819787081e1" member_type: VOTER last_known_addr { host: "127.3.89.252" port: 35283 } } }
I20260809 15:40:15.455216 27790 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1d43cc5aa2554fdf851c3663c9717f3c [sys.catalog]: This master's current role is: FOLLOWER
I20260809 15:40:15.456012 27819 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 15:40:15.462990 27731 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 9b24f2abb5ef4f7d86326819787081e1 [term 1 FOLLOWER]: Refusing update from remote peer 76ee3afa6e0e4189a630938d1e316744: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 15:40:15.463738 27772 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 76ee3afa6e0e4189a630938d1e316744 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9b24f2abb5ef4f7d86326819787081e1" member_type: VOTER last_known_addr { host: "127.3.89.252" port: 35283 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 15:40:15.468055 27771 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9b24f2abb5ef4f7d86326819787081e1 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 76ee3afa6e0e4189a630938d1e316744. Latest consensus state: current_term: 1 leader_uuid: "76ee3afa6e0e4189a630938d1e316744" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "76ee3afa6e0e4189a630938d1e316744" member_type: VOTER last_known_addr { host: "127.3.89.254" port: 33105 } } peers { permanent_uuid: "1d43cc5aa2554fdf851c3663c9717f3c" member_type: VOTER last_known_addr { host: "127.3.89.253" port: 37127 } } peers { permanent_uuid: "9b24f2abb5ef4f7d86326819787081e1" member_type: VOTER last_known_addr { host: "127.3.89.252" port: 35283 } } }
I20260809 15:40:15.468186 27771 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9b24f2abb5ef4f7d86326819787081e1 [sys.catalog]: This master's current role is: FOLLOWER
I20260809 15:40:15.468525 27649 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 1d43cc5aa2554fdf851c3663c9717f3c [term 1 FOLLOWER]: Refusing update from remote peer 76ee3afa6e0e4189a630938d1e316744: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 15:40:15.469218 27772 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 76ee3afa6e0e4189a630938d1e316744 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1d43cc5aa2554fdf851c3663c9717f3c" member_type: VOTER last_known_addr { host: "127.3.89.253" port: 37127 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 15:40:15.473049 27790 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1d43cc5aa2554fdf851c3663c9717f3c [sys.catalog]: SysCatalogTable state changed. Reason: New leader 76ee3afa6e0e4189a630938d1e316744. Latest consensus state: current_term: 1 leader_uuid: "76ee3afa6e0e4189a630938d1e316744" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "76ee3afa6e0e4189a630938d1e316744" member_type: VOTER last_known_addr { host: "127.3.89.254" port: 33105 } } peers { permanent_uuid: "1d43cc5aa2554fdf851c3663c9717f3c" member_type: VOTER last_known_addr { host: "127.3.89.253" port: 37127 } } peers { permanent_uuid: "9b24f2abb5ef4f7d86326819787081e1" member_type: VOTER last_known_addr { host: "127.3.89.252" port: 35283 } } }
I20260809 15:40:15.473183 27790 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1d43cc5aa2554fdf851c3663c9717f3c [sys.catalog]: This master's current role is: FOLLOWER
I20260809 15:40:15.477818 27817 sys_catalog.cc:455] T 00000000000000000000000000000000 P 76ee3afa6e0e4189a630938d1e316744 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "76ee3afa6e0e4189a630938d1e316744" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "76ee3afa6e0e4189a630938d1e316744" member_type: VOTER last_known_addr { host: "127.3.89.254" port: 33105 } } peers { permanent_uuid: "1d43cc5aa2554fdf851c3663c9717f3c" member_type: VOTER last_known_addr { host: "127.3.89.253" port: 37127 } } peers { permanent_uuid: "9b24f2abb5ef4f7d86326819787081e1" member_type: VOTER last_known_addr { host: "127.3.89.252" port: 35283 } } }
I20260809 15:40:15.477983 27817 sys_catalog.cc:458] T 00000000000000000000000000000000 P 76ee3afa6e0e4189a630938d1e316744 [sys.catalog]: This master's current role is: LEADER
I20260809 15:40:15.479549 27771 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9b24f2abb5ef4f7d86326819787081e1 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "76ee3afa6e0e4189a630938d1e316744" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "76ee3afa6e0e4189a630938d1e316744" member_type: VOTER last_known_addr { host: "127.3.89.254" port: 33105 } } peers { permanent_uuid: "1d43cc5aa2554fdf851c3663c9717f3c" member_type: VOTER last_known_addr { host: "127.3.89.253" port: 37127 } } peers { permanent_uuid: "9b24f2abb5ef4f7d86326819787081e1" member_type: VOTER last_known_addr { host: "127.3.89.252" port: 35283 } } }
I20260809 15:40:15.479682 27771 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9b24f2abb5ef4f7d86326819787081e1 [sys.catalog]: This master's current role is: FOLLOWER
I20260809 15:40:15.480206 27825 mvcc.cc:205] Tried to move back new op lower bound from 7316643903328759808 to 7316643903296618496. Current Snapshot: MvccSnapshot[applied={T|T < 7316643903328759808}]
I20260809 15:40:15.480646 27826 mvcc.cc:205] Tried to move back new op lower bound from 7316643903328759808 to 7316643903296618496. Current Snapshot: MvccSnapshot[applied={T|T < 7316643903328759808}]
I20260809 15:40:15.482187 27772 sys_catalog.cc:455] T 00000000000000000000000000000000 P 76ee3afa6e0e4189a630938d1e316744 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "76ee3afa6e0e4189a630938d1e316744" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "76ee3afa6e0e4189a630938d1e316744" member_type: VOTER last_known_addr { host: "127.3.89.254" port: 33105 } } peers { permanent_uuid: "1d43cc5aa2554fdf851c3663c9717f3c" member_type: VOTER last_known_addr { host: "127.3.89.253" port: 37127 } } peers { permanent_uuid: "9b24f2abb5ef4f7d86326819787081e1" member_type: VOTER last_known_addr { host: "127.3.89.252" port: 35283 } } }
I20260809 15:40:15.482302 27772 sys_catalog.cc:458] T 00000000000000000000000000000000 P 76ee3afa6e0e4189a630938d1e316744 [sys.catalog]: This master's current role is: LEADER
I20260809 15:40:15.483750 27790 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1d43cc5aa2554fdf851c3663c9717f3c [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "76ee3afa6e0e4189a630938d1e316744" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "76ee3afa6e0e4189a630938d1e316744" member_type: VOTER last_known_addr { host: "127.3.89.254" port: 33105 } } peers { permanent_uuid: "1d43cc5aa2554fdf851c3663c9717f3c" member_type: VOTER last_known_addr { host: "127.3.89.253" port: 37127 } } peers { permanent_uuid: "9b24f2abb5ef4f7d86326819787081e1" member_type: VOTER last_known_addr { host: "127.3.89.252" port: 35283 } } }
I20260809 15:40:15.483865 27790 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1d43cc5aa2554fdf851c3663c9717f3c [sys.catalog]: This master's current role is: FOLLOWER
I20260809 15:40:15.484355 27828 mvcc.cc:205] Tried to move back new op lower bound from 7316643903328759808 to 7316643903296618496. Current Snapshot: MvccSnapshot[applied={T|T < 7316643903328759808}]
I20260809 15:40:15.487262 27765 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:40:15.456604 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50356 (local address 127.3.89.252:35283)
0809 15:40:15.456711 (+ 107us) server_negotiation.cc:206] Beginning negotiation
0809 15:40:15.456715 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 15:40:15.463843 (+ 7128us) server_negotiation.cc:407] Connection header received
0809 15:40:15.464981 (+ 1138us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 15:40:15.464984 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 15:40:15.465031 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 15:40:15.465080 (+ 49us) server_negotiation.cc:226] Negotiated authn=SASL
0809 15:40:15.468038 (+ 2958us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:40:15.468827 (+ 789us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:40:15.477289 (+ 8462us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:40:15.477650 (+ 361us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:40:15.477723 (+ 73us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:40:15.481743 (+ 4020us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 15:40:15.481747 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 15:40:15.481750 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 15:40:15.481777 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0809 15:40:15.485092 (+ 3315us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 15:40:15.485917 (+ 825us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 15:40:15.485921 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 15:40:15.485923 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 15:40:15.486009 (+ 86us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 15:40:15.486298 (+ 289us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 15:40:15.486302 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 15:40:15.486304 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 15:40:15.486521 (+ 217us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 15:40:15.486631 (+ 110us) server_negotiation.cc:1035] Waiting for connection context
0809 15:40:15.486985 (+ 354us) server_negotiation.cc:299] Negotiation successful
0809 15:40:15.487106 (+ 121us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":33}
I20260809 15:40:15.487361 27600 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:40:15.455819 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45272 (local address 127.3.89.254:33105)
0809 15:40:15.455945 (+ 126us) server_negotiation.cc:206] Beginning negotiation
0809 15:40:15.455949 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 15:40:15.463238 (+ 7289us) server_negotiation.cc:407] Connection header received
0809 15:40:15.463461 (+ 223us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 15:40:15.463465 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 15:40:15.463514 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 15:40:15.463566 (+ 52us) server_negotiation.cc:226] Negotiated authn=SASL
0809 15:40:15.466018 (+ 2452us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:40:15.466811 (+ 793us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:40:15.475188 (+ 8377us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:40:15.475562 (+ 374us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:40:15.475638 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:40:15.482142 (+ 6504us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 15:40:15.482145 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 15:40:15.482149 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 15:40:15.482178 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0809 15:40:15.484408 (+ 2230us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 15:40:15.485917 (+ 1509us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 15:40:15.485921 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 15:40:15.485923 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 15:40:15.486018 (+ 95us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 15:40:15.486342 (+ 324us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 15:40:15.486346 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 15:40:15.486347 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 15:40:15.486575 (+ 228us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 15:40:15.486682 (+ 107us) server_negotiation.cc:1035] Waiting for connection context
0809 15:40:15.487065 (+ 383us) server_negotiation.cc:299] Negotiation successful
0809 15:40:15.487214 (+ 149us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":45}
I20260809 15:40:15.489020 27819 catalog_manager.cc:1347] Generated new cluster ID: 28c1f177ff0441f5aa1d51368638e1f5
I20260809 15:40:15.489082 27819 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 15:40:15.492477 27683 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:40:15.455362 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47360 (local address 127.3.89.253:37127)
0809 15:40:15.455475 (+ 113us) server_negotiation.cc:206] Beginning negotiation
0809 15:40:15.455479 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 15:40:15.470181 (+ 14702us) server_negotiation.cc:407] Connection header received
0809 15:40:15.476196 (+ 6015us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 15:40:15.476200 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 15:40:15.476254 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 15:40:15.476306 (+ 52us) server_negotiation.cc:226] Negotiated authn=SASL
0809 15:40:15.478214 (+ 1908us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:40:15.479043 (+ 829us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:40:15.480181 (+ 1138us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:40:15.480519 (+ 338us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:40:15.480590 (+ 71us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:40:15.489039 (+ 8449us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 15:40:15.489043 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 15:40:15.489046 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 15:40:15.489073 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0809 15:40:15.490748 (+ 1675us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 15:40:15.491240 (+ 492us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 15:40:15.491243 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 15:40:15.491245 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 15:40:15.491322 (+ 77us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 15:40:15.491598 (+ 276us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 15:40:15.491601 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 15:40:15.491603 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 15:40:15.491798 (+ 195us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 15:40:15.491894 (+ 96us) server_negotiation.cc:1035] Waiting for connection context
0809 15:40:15.492197 (+ 303us) server_negotiation.cc:299] Negotiation successful
0809 15:40:15.492314 (+ 117us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":41}
I20260809 15:40:15.512609 27819 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 15:40:15.513947 27819 catalog_manager.cc:1504] Loading token signing keys...
I20260809 15:40:15.535212 27819 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 76ee3afa6e0e4189a630938d1e316744: Generated new TSK 0
I20260809 15:40:15.536262 27819 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260809 15:40:15.567641 27600 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 15:40:15.564516 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45278 (local address 127.3.89.254:33105)
0809 15:40:15.564642 (+ 126us) server_negotiation.cc:206] Beginning negotiation
0809 15:40:15.564647 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 15:40:15.566507 (+ 1860us) server_negotiation.cc:407] Connection header received
0809 15:40:15.566753 (+ 246us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 15:40:15.566757 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 15:40:15.566805 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 15:40:15.566859 (+ 54us) server_negotiation.cc:226] Negotiated authn=SASL
0809 15:40:15.567510 (+ 651us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:45278: BlockingRecv error: recv got EOF from 127.0.0.1:45278 (error 108)
Metrics: {"server-negotiator.queue_time_us":50}
W20260809 15:40:15.569237 27765 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 15:40:15.565218 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50360 (local address 127.3.89.252:35283)
0809 15:40:15.565356 (+ 138us) server_negotiation.cc:206] Beginning negotiation
0809 15:40:15.565361 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 15:40:15.568016 (+ 2655us) server_negotiation.cc:407] Connection header received
0809 15:40:15.568512 (+ 496us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 15:40:15.568515 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 15:40:15.568561 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 15:40:15.568612 (+ 51us) server_negotiation.cc:226] Negotiated authn=SASL
0809 15:40:15.569127 (+ 515us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:50360: BlockingRecv error: recv got EOF from 127.0.0.1:50360 (error 108)
Metrics: {"server-negotiator.queue_time_us":44}
W20260809 15:40:15.570254 27683 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 15:40:15.565874 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47376 (local address 127.3.89.253:37127)
0809 15:40:15.566021 (+ 147us) server_negotiation.cc:206] Beginning negotiation
0809 15:40:15.566026 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 15:40:15.568434 (+ 2408us) server_negotiation.cc:407] Connection header received
0809 15:40:15.569596 (+ 1162us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 15:40:15.569599 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 15:40:15.569646 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 15:40:15.569695 (+ 49us) server_negotiation.cc:226] Negotiated authn=SASL
0809 15:40:15.570146 (+ 451us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:47376: BlockingRecv error: recv got EOF from 127.0.0.1:47376 (error 108)
Metrics: {"server-negotiator.queue_time_us":45}
W20260809 15:40:15.576712 27683 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 15:40:15.575342 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47382 (local address 127.3.89.253:37127)
0809 15:40:15.575438 (+ 96us) server_negotiation.cc:206] Beginning negotiation
0809 15:40:15.575441 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0809 15:40:15.575866 (+ 425us) server_negotiation.cc:407] Connection header received
0809 15:40:15.576058 (+ 192us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 15:40:15.576061 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 15:40:15.576127 (+ 66us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 15:40:15.576176 (+ 49us) server_negotiation.cc:226] Negotiated authn=SASL
0809 15:40:15.576623 (+ 447us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:47382: BlockingRecv error: recv got EOF from 127.0.0.1:47382 (error 108)
Metrics: {"server-negotiator.queue_time_us":38}
W20260809 15:40:15.585299 27683 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 15:40:15.583729 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47396 (local address 127.3.89.253:37127)
0809 15:40:15.583834 (+ 105us) server_negotiation.cc:206] Beginning negotiation
0809 15:40:15.583838 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 15:40:15.584373 (+ 535us) server_negotiation.cc:407] Connection header received
0809 15:40:15.584585 (+ 212us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 15:40:15.584591 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 15:40:15.584644 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 15:40:15.584698 (+ 54us) server_negotiation.cc:226] Negotiated authn=SASL
0809 15:40:15.585210 (+ 512us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:47396: BlockingRecv error: recv got EOF from 127.0.0.1:47396 (error 108)
Metrics: {"server-negotiator.queue_time_us":39}
W20260809 15:40:15.591050 27683 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 15:40:15.589497 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47398 (local address 127.3.89.253:37127)
0809 15:40:15.589597 (+ 100us) server_negotiation.cc:206] Beginning negotiation
0809 15:40:15.589601 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 15:40:15.590107 (+ 506us) server_negotiation.cc:407] Connection header received
0809 15:40:15.590289 (+ 182us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 15:40:15.590293 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 15:40:15.590354 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 15:40:15.590406 (+ 52us) server_negotiation.cc:226] Negotiated authn=SASL
0809 15:40:15.590890 (+ 484us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:47398: BlockingRecv error: recv got EOF from 127.0.0.1:47398 (error 108)
Metrics: {"server-negotiator.queue_time_us":41}
W20260809 15:40:15.599161 27683 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 15:40:15.597476 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47400 (local address 127.3.89.253:37127)
0809 15:40:15.597616 (+ 140us) server_negotiation.cc:206] Beginning negotiation
0809 15:40:15.597621 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 15:40:15.598189 (+ 568us) server_negotiation.cc:407] Connection header received
0809 15:40:15.598382 (+ 193us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 15:40:15.598385 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 15:40:15.598436 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 15:40:15.598494 (+ 58us) server_negotiation.cc:226] Negotiated authn=SASL
0809 15:40:15.599066 (+ 572us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:47400: BlockingRecv error: recv got EOF from 127.0.0.1:47400 (error 108)
Metrics: {"server-negotiator.queue_time_us":67}
W20260809 15:40:15.611325 27683 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 15:40:15.609525 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47414 (local address 127.3.89.253:37127)
0809 15:40:15.609645 (+ 120us) server_negotiation.cc:206] Beginning negotiation
0809 15:40:15.609650 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 15:40:15.610250 (+ 600us) server_negotiation.cc:407] Connection header received
0809 15:40:15.610478 (+ 228us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 15:40:15.610482 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 15:40:15.610535 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 15:40:15.610589 (+ 54us) server_negotiation.cc:226] Negotiated authn=SASL
0809 15:40:15.611201 (+ 612us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:47414: BlockingRecv error: recv got EOF from 127.0.0.1:47414 (error 108)
Metrics: {"server-negotiator.queue_time_us":47}
W20260809 15:40:15.631742 27683 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 15:40:15.629929 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47428 (local address 127.3.89.253:37127)
0809 15:40:15.630055 (+ 126us) server_negotiation.cc:206] Beginning negotiation
0809 15:40:15.630061 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 15:40:15.630659 (+ 598us) server_negotiation.cc:407] Connection header received
0809 15:40:15.630865 (+ 206us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 15:40:15.630869 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 15:40:15.630924 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 15:40:15.631020 (+ 96us) server_negotiation.cc:226] Negotiated authn=SASL
0809 15:40:15.631625 (+ 605us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:47428: BlockingRecv error: recv got EOF from 127.0.0.1:47428 (error 108)
Metrics: {"server-negotiator.queue_time_us":49}
W20260809 15:40:15.667670 27683 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 15:40:15.666358 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47440 (local address 127.3.89.253:37127)
0809 15:40:15.666476 (+ 118us) server_negotiation.cc:206] Beginning negotiation
0809 15:40:15.666479 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0809 15:40:15.666673 (+ 194us) server_negotiation.cc:407] Connection header received
0809 15:40:15.666834 (+ 161us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 15:40:15.666836 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 15:40:15.666884 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 15:40:15.667002 (+ 118us) server_negotiation.cc:226] Negotiated authn=SASL
0809 15:40:15.667540 (+ 538us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:47440: BlockingRecv error: recv got EOF from 127.0.0.1:47440 (error 108)
Metrics: {"server-negotiator.queue_time_us":34}
W20260809 15:40:15.735291 27683 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 15:40:15.734048 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47442 (local address 127.3.89.253:37127)
0809 15:40:15.734203 (+ 155us) server_negotiation.cc:206] Beginning negotiation
0809 15:40:15.734208 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 15:40:15.734400 (+ 192us) server_negotiation.cc:407] Connection header received
0809 15:40:15.734513 (+ 113us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 15:40:15.734516 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 15:40:15.734565 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 15:40:15.734639 (+ 74us) server_negotiation.cc:226] Negotiated authn=SASL
0809 15:40:15.735147 (+ 508us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:47442: BlockingRecv error: recv got EOF from 127.0.0.1:47442 (error 108)
Metrics: {"server-negotiator.queue_time_us":60}
W20260809 15:40:15.866809 27683 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 15:40:15.865619 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47452 (local address 127.3.89.253:37127)
0809 15:40:15.865826 (+ 207us) server_negotiation.cc:206] Beginning negotiation
0809 15:40:15.865830 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 15:40:15.865961 (+ 131us) server_negotiation.cc:407] Connection header received
0809 15:40:15.866088 (+ 127us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 15:40:15.866091 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 15:40:15.866140 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 15:40:15.866213 (+ 73us) server_negotiation.cc:226] Negotiated authn=SASL
0809 15:40:15.866683 (+ 470us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:47452: BlockingRecv error: recv got EOF from 127.0.0.1:47452 (error 108)
Metrics: {"server-negotiator.queue_time_us":104}
W20260809 15:40:16.126353 27683 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 15:40:16.125194 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47454 (local address 127.3.89.253:37127)
0809 15:40:16.125333 (+ 139us) server_negotiation.cc:206] Beginning negotiation
0809 15:40:16.125336 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0809 15:40:16.125407 (+ 71us) server_negotiation.cc:407] Connection header received
0809 15:40:16.125535 (+ 128us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 15:40:16.125538 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 15:40:16.125586 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 15:40:16.125670 (+ 84us) server_negotiation.cc:226] Negotiated authn=SASL
0809 15:40:16.126226 (+ 556us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:47454: BlockingRecv error: recv got EOF from 127.0.0.1:47454 (error 108)
Metrics: {"server-negotiator.queue_time_us":57}
I20260809 15:40:16.437402 27789 catalog_manager.cc:1259] Loaded cluster ID: 28c1f177ff0441f5aa1d51368638e1f5
I20260809 15:40:16.437484 27789 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 9b24f2abb5ef4f7d86326819787081e1: loading cluster ID for follower catalog manager: success
I20260809 15:40:16.439210 27789 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 9b24f2abb5ef4f7d86326819787081e1: acquiring CA information for follower catalog manager: success
I20260809 15:40:16.440402 27789 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 9b24f2abb5ef4f7d86326819787081e1: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260809 15:40:16.445766 27804 catalog_manager.cc:1259] Loaded cluster ID: 28c1f177ff0441f5aa1d51368638e1f5
I20260809 15:40:16.445861 27804 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 1d43cc5aa2554fdf851c3663c9717f3c: loading cluster ID for follower catalog manager: success
I20260809 15:40:16.447554 27804 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 1d43cc5aa2554fdf851c3663c9717f3c: acquiring CA information for follower catalog manager: success
I20260809 15:40:16.448774 27804 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 1d43cc5aa2554fdf851c3663c9717f3c: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260809 15:40:16.644771 27900 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:40:16.640696 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47462 (local address 127.3.89.253:37127)
0809 15:40:16.641023 (+ 327us) server_negotiation.cc:206] Beginning negotiation
0809 15:40:16.641028 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 15:40:16.641046 (+ 18us) server_negotiation.cc:407] Connection header received
0809 15:40:16.641207 (+ 161us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 15:40:16.641218 (+ 11us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 15:40:16.641346 (+ 128us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 15:40:16.641468 (+ 122us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 15:40:16.641772 (+ 304us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:40:16.642305 (+ 533us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:40:16.643275 (+ 970us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:40:16.643528 (+ 253us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:40:16.643618 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:40:16.643897 (+ 279us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 15:40:16.644343 (+ 446us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 15:40:16.644438 (+ 95us) server_negotiation.cc:1035] Waiting for connection context
0809 15:40:16.644513 (+ 75us) server_negotiation.cc:299] Negotiation successful
0809 15:40:16.644565 (+ 52us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":216,"thread_start_us":108,"threads_started":1}
I20260809 15:40:16.651340 27909 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:40:16.647844 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50370 (local address 127.3.89.252:35283)
0809 15:40:16.648072 (+ 228us) server_negotiation.cc:206] Beginning negotiation
0809 15:40:16.648075 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0809 15:40:16.648087 (+ 12us) server_negotiation.cc:407] Connection header received
0809 15:40:16.648161 (+ 74us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 15:40:16.648163 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 15:40:16.648265 (+ 102us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 15:40:16.648364 (+ 99us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 15:40:16.648546 (+ 182us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:40:16.649153 (+ 607us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:40:16.649929 (+ 776us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:40:16.650164 (+ 235us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:40:16.650256 (+ 92us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:40:16.650491 (+ 235us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 15:40:16.650931 (+ 440us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 15:40:16.651090 (+ 159us) server_negotiation.cc:1035] Waiting for connection context
0809 15:40:16.651118 (+ 28us) server_negotiation.cc:299] Negotiation successful
0809 15:40:16.651161 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":167,"thread_start_us":78,"threads_started":1}
I20260809 15:40:16.658435 27917 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:40:16.654339 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45290 (local address 127.3.89.254:33105)
0809 15:40:16.654607 (+ 268us) server_negotiation.cc:206] Beginning negotiation
0809 15:40:16.654612 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 15:40:16.654627 (+ 15us) server_negotiation.cc:407] Connection header received
0809 15:40:16.654705 (+ 78us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 15:40:16.654707 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 15:40:16.654823 (+ 116us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 15:40:16.654950 (+ 127us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 15:40:16.655216 (+ 266us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:40:16.655877 (+ 661us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:40:16.656878 (+ 1001us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:40:16.657242 (+ 364us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:40:16.657353 (+ 111us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:40:16.657680 (+ 327us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 15:40:16.658000 (+ 320us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 15:40:16.658091 (+ 91us) server_negotiation.cc:1035] Waiting for connection context
0809 15:40:16.658224 (+ 133us) server_negotiation.cc:299] Negotiation successful
0809 15:40:16.658280 (+ 56us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":197,"thread_start_us":85,"threads_started":1}
I20260809 15:40:16.658625 27909 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:40:16.654441 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50372 (local address 127.3.89.252:35283)
0809 15:40:16.654555 (+ 114us) server_negotiation.cc:206] Beginning negotiation
0809 15:40:16.654560 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 15:40:16.655091 (+ 531us) server_negotiation.cc:407] Connection header received
0809 15:40:16.655145 (+ 54us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 15:40:16.655150 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 15:40:16.655203 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 15:40:16.655269 (+ 66us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 15:40:16.655523 (+ 254us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:40:16.656242 (+ 719us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:40:16.657258 (+ 1016us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:40:16.657606 (+ 348us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:40:16.657706 (+ 100us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:40:16.657965 (+ 259us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 15:40:16.658036 (+ 71us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 15:40:16.658132 (+ 96us) server_negotiation.cc:1035] Waiting for connection context
0809 15:40:16.658432 (+ 300us) server_negotiation.cc:299] Negotiation successful
0809 15:40:16.658485 (+ 53us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":41}
I20260809 15:40:16.658784 27900 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:40:16.654097 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47478 (local address 127.3.89.253:37127)
0809 15:40:16.654252 (+ 155us) server_negotiation.cc:206] Beginning negotiation
0809 15:40:16.654257 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 15:40:16.655018 (+ 761us) server_negotiation.cc:407] Connection header received
0809 15:40:16.655216 (+ 198us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 15:40:16.655219 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 15:40:16.655272 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 15:40:16.655344 (+ 72us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 15:40:16.655614 (+ 270us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:40:16.656333 (+ 719us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:40:16.657480 (+ 1147us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:40:16.657742 (+ 262us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:40:16.657853 (+ 111us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:40:16.658126 (+ 273us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 15:40:16.658200 (+ 74us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 15:40:16.658266 (+ 66us) server_negotiation.cc:1035] Waiting for connection context
0809 15:40:16.658581 (+ 315us) server_negotiation.cc:299] Negotiation successful
0809 15:40:16.658633 (+ 52us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":59}
I20260809 15:40:16.661799 3431 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskm314YR/build/debug/bin/kudu with pid 27523
I20260809 15:40:16.670715 3431 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskm314YR/build/debug/bin/kudu with pid 27605
I20260809 15:40:16.678715 3431 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskm314YR/build/debug/bin/kudu with pid 27685
2026-08-09T15:40:16Z chronyd exiting
[ OK ] ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys/0 (3131 ms)
[ RUN ] ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys/1
Loading random data
Initializing database '/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786289732823074-3431-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 15:40:16 dist-test-slave-2w3w krb5kdc[27921](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 15:40:16 dist-test-slave-2w3w krb5kdc[27921](info): set up 2 sockets
Aug 09 15:40:16 dist-test-slave-2w3w krb5kdc[27921](info): commencing operation
krb5kdc: starting...
W20260809 15:40:17.769109 3431 mini_kdc.cc:121] Time spent starting KDC: real 1.067s user 0.000s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786289732823074-3431-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786289732823074-3431-0/krb5kdc/test-user.keytab.
Aug 09 15:40:17 dist-test-slave-2w3w krb5kdc[27921](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786290017, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T15:40:17Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T15:40:17Z Disabled control of system clock
WARNING: no policy specified for kudu/127.3.89.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.3.89.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.3.89.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786289732823074-3431-0/krb5kdc/kudu_127.3.89.254.keytab.
Entry for principal kudu/127.3.89.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786289732823074-3431-0/krb5kdc/kudu_127.3.89.254.keytab.
I20260809 15:40:17.897858 3431 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskm314YR/build/debug/bin/kudu
/tmp/dist-test-taskm314YR/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786289732823074-3431-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786289732823074-3431-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786289732823074-3431-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786289732823074-3431-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.3.89.254:32885
--webserver_interface=127.3.89.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786289732823074-3431-0/krb5kdc/kudu_127.3.89.254.keytab
--principal=kudu/127.3.89.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.3.89.212:43203
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.3.89.254:32885,127.3.89.253:46795,127.3.89.252:39119
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=required with env {KRB5CCNAME=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786289732823074-3431-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786289732823074-3431-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786289732823074-3431-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 15:40:18.008538 27934 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260809 15:40:18.008884 27934 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 15:40:18.008948 27934 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 15:40:18.009027 27934 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 15:40:18.013198 27934 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 15:40:18.013288 27934 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 15:40:18.013331 27934 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 15:40:18.013414 27934 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 15:40:18.013451 27934 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 15:40:18.018718 27934 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.3.89.212:43203
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786289732823074-3431-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786289732823074-3431-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.3.89.254:32885,127.3.89.253:46795,127.3.89.252:39119
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786289732823074-3431-0/krb5kdc/kudu_127.3.89.254.keytab
--principal=kudu/127.3.89.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.3.89.254:32885
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786289732823074-3431-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.3.89.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.27934
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786289732823074-3431-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 09 Aug 2026 14:46:55 UTC on ccycloud.quasar-yswkmf.root.comops.site
build id 4576540
I20260809 15:40:18.019809 27934 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 15:40:18.020823 27934 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 15:40:18.029127 27940 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 15:40:18.029259 27934 server_base.cc:1034] running on GCE node
W20260809 15:40:18.029119 27941 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 15:40:18.029419 27944 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 15:40:18.029980 27934 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 15:40:18.031149 27934 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 15:40:18.032351 27934 hybrid_clock.cc:648] HybridClock initialized: now 1786290018032320 us; error 42 us; skew 500 ppm
Aug 09 15:40:18 dist-test-slave-2w3w krb5kdc[27921](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786290018, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.89.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 15:40:18.035207 27934 init.cc:377] Logged in from keytab as kudu/127.3.89.254@KRBTEST.COM (short username kudu)
I20260809 15:40:18.036170 27934 webserver.cc:460] Webserver started at http://127.3.89.254:39111/ using document root <none> and password file <none>
I20260809 15:40:18.036712 27934 fs_manager.cc:359] Metadata directory not provided
I20260809 15:40:18.036816 27934 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 15:40:18.037043 27934 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 15:40:18.039108 27934 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskm314YR/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786289732823074-3431-0/minicluster-data/master-0/data/instance:
uuid: "8ba587d8bdca4e27ac7a45bbf5f8a260"
format_stamp: "Formatted at 2026-08-09 15:40:18 on dist-test-slave-2w3w"
server_key: "eed5b4d9a8b07a067aba411f82294e52"
server_key_iv: "3b5b13d17a4f274a2b068625a4995ef6"
server_key_version: "encryptionkey@0"
I20260809 15:40:18.039674 27934 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskm314YR/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786289732823074-3431-0/minicluster-data/master-0/wal/instance:
uuid: "8ba587d8bdca4e27ac7a45bbf5f8a260"
format_stamp: "Formatted at 2026-08-09 15:40:18 on dist-test-slave-2w3w"
server_key: "eed5b4d9a8b07a067aba411f82294e52"
server_key_iv: "3b5b13d17a4f274a2b068625a4995ef6"
server_key_version: "encryptionkey@0"
I20260809 15:40:18.043428 27934 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.003s sys 0.000s
I20260809 15:40:18.045786 27951 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 15:40:18.046927 27934 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260809 15:40:18.047132 27934 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskm314YR/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786289732823074-3431-0/minicluster-data/master-0/data,/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786289732823074-3431-0/minicluster-data/master-0/wal
uuid: "8ba587d8bdca4e27ac7a45bbf5f8a260"
format_stamp: "Formatted at 2026-08-09 15:40:18 on dist-test-slave-2w3w"
server_key: "eed5b4d9a8b07a067aba411f82294e52"
server_key_iv: "3b5b13d17a4f274a2b068625a4995ef6"
server_key_version: "encryptionkey@0"
I20260809 15:40:18.047243 27934 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskm314YR/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786289732823074-3431-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskm314YR/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786289732823074-3431-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskm314YR/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786289732823074-3431-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 15:40:18.060191 27934 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 15:40:18.062987 27934 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 15:40:18.063192 27934 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 15:40:18.070159 27934 rpc_server.cc:307] RPC server started. Bound to: 127.3.89.254:32885
I20260809 15:40:18.070446 28008 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.3.89.254:32885 every 8 connection(s)
I20260809 15:40:18.071158 27934 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskm314YR/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786289732823074-3431-0/minicluster-data/master-0/data/info.pb
I20260809 15:40:18.073777 28009 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 15:40:18.074995 3431 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskm314YR/build/debug/bin/kudu as pid 27934
I20260809 15:40:18.075122 3431 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskm314YR/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786289732823074-3431-0/minicluster-data/master-0/wal/instance
I20260809 15:40:18.075476 3431 external_mini_cluster.cc:1428] Setting key c4ff9ef3829a502c50906b35a8036478
I20260809 15:40:18.078711 28009 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.3.89.254" port: 32885 } has no permanent_uuid. Determining permanent_uuid...
WARNING: no policy specified for kudu/127.3.89.253@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.3.89.253@KRBTEST.COM" created.
Aug 09 15:40:18 dist-test-slave-2w3w krb5kdc[27921](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786290018, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.89.254@KRBTEST.COM for kudu/127.3.89.254@KRBTEST.COM
I20260809 15:40:18.094885 28014 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:40:18.081403 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.89.254:32885 (local address 127.0.0.1:34414)
0809 15:40:18.082021 (+ 618us) negotiation.cc:107] Waiting for socket to connect
0809 15:40:18.082068 (+ 47us) client_negotiation.cc:174] Beginning negotiation
0809 15:40:18.082867 (+ 799us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 15:40:18.083730 (+ 863us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 15:40:18.083734 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 15:40:18.084060 (+ 326us) client_negotiation.cc:189] Negotiated authn=SASL
0809 15:40:18.084363 (+ 303us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:40:18.084376 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:40:18.085879 (+ 1503us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:40:18.085882 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:40:18.086743 (+ 861us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:40:18.086752 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:40:18.087453 (+ 701us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:40:18.087456 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:40:18.087552 (+ 96us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:40:18.088156 (+ 604us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 15:40:18.088172 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0809 15:40:18.089858 (+ 1686us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 15:40:18.091855 (+ 1997us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 15:40:18.091865 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 15:40:18.091874 (+ 9us) client_negotiation.cc:760] calling sasl_client_step()
0809 15:40:18.092117 (+ 243us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 15:40:18.092451 (+ 334us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 15:40:18.092453 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 15:40:18.092456 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0809 15:40:18.092571 (+ 115us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 15:40:18.093162 (+ 591us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 15:40:18.093168 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 15:40:18.093308 (+ 140us) client_negotiation.cc:769] Sending connection context
0809 15:40:18.093481 (+ 173us) client_negotiation.cc:240] Negotiation successful
0809 15:40:18.093862 (+ 381us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":445,"mutex_wait_us":132,"spinlock_wait_cycles":48512,"thread_start_us":156,"threads_started":1}
I20260809 15:40:18.094885 28012 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:40:18.081403 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34414 (local address 127.3.89.254:32885)
0809 15:40:18.081923 (+ 520us) server_negotiation.cc:206] Beginning negotiation
0809 15:40:18.081944 (+ 21us) server_negotiation.cc:399] Waiting for connection header
0809 15:40:18.082190 (+ 246us) server_negotiation.cc:407] Connection header received
0809 15:40:18.083236 (+ 1046us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 15:40:18.083245 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 15:40:18.083510 (+ 265us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 15:40:18.083722 (+ 212us) server_negotiation.cc:226] Negotiated authn=SASL
0809 15:40:18.084527 (+ 805us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:40:18.085762 (+ 1235us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:40:18.086859 (+ 1097us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:40:18.087282 (+ 423us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:40:18.087404 (+ 122us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:40:18.090025 (+ 2621us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 15:40:18.090031 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 15:40:18.090040 (+ 9us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 15:40:18.090060 (+ 20us) server_negotiation.cc:922] calling sasl_server_start()
0809 15:40:18.091724 (+ 1664us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 15:40:18.092233 (+ 509us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 15:40:18.092239 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 15:40:18.092240 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 15:40:18.092349 (+ 109us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 15:40:18.092663 (+ 314us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 15:40:18.092668 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 15:40:18.092670 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 15:40:18.093036 (+ 366us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 15:40:18.093177 (+ 141us) server_negotiation.cc:1035] Waiting for connection context
0809 15:40:18.093563 (+ 386us) server_negotiation.cc:299] Negotiation successful
0809 15:40:18.093808 (+ 245us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":33,"server-negotiator.queue_time_us":337,"thread_start_us":149,"threads_started":1}
I20260809 15:40:18.097580 28009 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.3.89.253" port: 46795 } has no permanent_uuid. Determining permanent_uuid...
I20260809 15:40:18.098342 28014 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:40:18.098039 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.89.253:46795 (local address 127.0.0.1:41564)
0809 15:40:18.098165 (+ 126us) negotiation.cc:107] Waiting for socket to connect
0809 15:40:18.098275 (+ 110us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.89.253:46795: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":52}
W20260809 15:40:18.098549 27955 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.3.89.253:46795: connect: Connection refused (error 111)
I20260809 15:40:18.099589 28014 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:40:18.099386 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.89.253:46795 (local address 127.0.0.1:41580)
0809 15:40:18.099489 (+ 103us) negotiation.cc:107] Waiting for socket to connect
0809 15:40:18.099535 (+ 46us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.89.253:46795: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":43}
W20260809 15:40:18.099728 28009 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.3.89.253:46795: Network error: Client connection negotiation failed: client connection to 127.3.89.253:46795: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.3.89.253 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786289732823074-3431-0/krb5kdc/kudu_127.3.89.253.keytab.
Entry for principal kudu/127.3.89.253 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786289732823074-3431-0/krb5kdc/kudu_127.3.89.253.keytab.
I20260809 15:40:18.106052 3431 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskm314YR/build/debug/bin/kudu
/tmp/dist-test-taskm314YR/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786289732823074-3431-0/minicluster-data/master-1/wal
--fs_data_dirs=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786289732823074-3431-0/minicluster-data/master-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786289732823074-3431-0/minicluster-data/master-1/logs
--server_dump_info_path=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786289732823074-3431-0/minicluster-data/master-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.3.89.253:46795
--webserver_interface=127.3.89.253
--webserver_port=0
--keytab_file=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786289732823074-3431-0/krb5kdc/kudu_127.3.89.253.keytab
--principal=kudu/127.3.89.253
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.3.89.212:43203
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.3.89.254:32885,127.3.89.253:46795,127.3.89.252:39119
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=required with env {KRB5CCNAME=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786289732823074-3431-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786289732823074-3431-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786289732823074-3431-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260809 15:40:18.149037 28009 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.3.89.253" port: 46795 } attempt: 1
I20260809 15:40:18.150018 28014 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:40:18.149671 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.89.253:46795 (local address 127.0.0.1:41588)
0809 15:40:18.149850 (+ 179us) negotiation.cc:107] Waiting for socket to connect
0809 15:40:18.149926 (+ 76us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.89.253:46795: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":66}
I20260809 15:40:18.150799 28014 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:40:18.150574 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.89.253:46795 (local address 127.0.0.1:41598)
0809 15:40:18.150660 (+ 86us) negotiation.cc:107] Waiting for socket to connect
0809 15:40:18.150733 (+ 73us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.89.253:46795: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":22}
W20260809 15:40:18.150950 28009 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.3.89.253:46795: Network error: Client connection negotiation failed: client connection to 127.3.89.253:46795: connect: Connection refused (error 111)
W20260809 15:40:18.217509 28018 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260809 15:40:18.217767 28018 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 15:40:18.217842 28018 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 15:40:18.217937 28018 flags.cc:425] Enabled unsafe flag: --never_fsync=true
I20260809 15:40:18.219167 28009 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.3.89.253" port: 46795 } attempt: 2
I20260809 15:40:18.219992 28014 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:40:18.219659 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.89.253:46795 (local address 127.0.0.1:41602)
0809 15:40:18.219833 (+ 174us) negotiation.cc:107] Waiting for socket to connect
0809 15:40:18.219904 (+ 71us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.89.253:46795: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":61}
I20260809 15:40:18.220743 28014 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:40:18.220521 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.89.253:46795 (local address 127.0.0.1:41616)
0809 15:40:18.220627 (+ 106us) negotiation.cc:107] Waiting for socket to connect
0809 15:40:18.220678 (+ 51us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.89.253:46795: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":40}
W20260809 15:40:18.220881 28009 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.3.89.253:46795: Network error: Client connection negotiation failed: client connection to 127.3.89.253:46795: connect: Connection refused (error 111)
W20260809 15:40:18.222487 28018 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 15:40:18.222564 28018 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 15:40:18.222608 28018 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 15:40:18.222645 28018 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 15:40:18.222680 28018 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 15:40:18.227861 28018 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.3.89.212:43203
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786289732823074-3431-0/minicluster-data/master-1/data
--fs_wal_dir=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786289732823074-3431-0/minicluster-data/master-1/wal
--ipki_ca_key_size=768
--master_addresses=127.3.89.254:32885,127.3.89.253:46795,127.3.89.252:39119
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786289732823074-3431-0/krb5kdc/kudu_127.3.89.253.keytab
--principal=kudu/127.3.89.253
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.3.89.253:46795
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786289732823074-3431-0/minicluster-data/master-1/data/info.pb
--webserver_interface=127.3.89.253
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.28018
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786289732823074-3431-0/minicluster-data/master-1/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 09 Aug 2026 14:46:55 UTC on ccycloud.quasar-yswkmf.root.comops.site
build id 4576540
I20260809 15:40:18.228870 28018 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 15:40:18.229837 28018 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 15:40:18.238127 28024 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 15:40:18.238130 28025 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 15:40:18.238130 28027 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 15:40:18.238389 28018 server_base.cc:1034] running on GCE node
I20260809 15:40:18.239041 28018 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 15:40:18.240068 28018 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 15:40:18.241258 28018 hybrid_clock.cc:648] HybridClock initialized: now 1786290018241244 us; error 29 us; skew 500 ppm
Aug 09 15:40:18 dist-test-slave-2w3w krb5kdc[27921](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786290018, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.89.253@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 15:40:18.244160 28018 init.cc:377] Logged in from keytab as kudu/127.3.89.253@KRBTEST.COM (short username kudu)
I20260809 15:40:18.245026 28018 webserver.cc:460] Webserver started at http://127.3.89.253:36209/ using document root <none> and password file <none>
I20260809 15:40:18.245573 28018 fs_manager.cc:359] Metadata directory not provided
I20260809 15:40:18.245666 28018 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 15:40:18.245898 28018 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 15:40:18.247939 28018 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskm314YR/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786289732823074-3431-0/minicluster-data/master-1/data/instance:
uuid: "c064bc9a409c478cb810307783d32488"
format_stamp: "Formatted at 2026-08-09 15:40:18 on dist-test-slave-2w3w"
server_key: "c44b5d3f464cecbae64fa53c14ebd2f7"
server_key_iv: "5aa16a210f3b3410f16a4d86ac4508ef"
server_key_version: "encryptionkey@0"
I20260809 15:40:18.248513 28018 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskm314YR/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786289732823074-3431-0/minicluster-data/master-1/wal/instance:
uuid: "c064bc9a409c478cb810307783d32488"
format_stamp: "Formatted at 2026-08-09 15:40:18 on dist-test-slave-2w3w"
server_key: "c44b5d3f464cecbae64fa53c14ebd2f7"
server_key_iv: "5aa16a210f3b3410f16a4d86ac4508ef"
server_key_version: "encryptionkey@0"
I20260809 15:40:18.252144 28018 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.004s sys 0.000s
I20260809 15:40:18.254362 28034 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 15:40:18.255548 28018 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260809 15:40:18.255692 28018 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskm314YR/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786289732823074-3431-0/minicluster-data/master-1/data,/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786289732823074-3431-0/minicluster-data/master-1/wal
uuid: "c064bc9a409c478cb810307783d32488"
format_stamp: "Formatted at 2026-08-09 15:40:18 on dist-test-slave-2w3w"
server_key: "c44b5d3f464cecbae64fa53c14ebd2f7"
server_key_iv: "5aa16a210f3b3410f16a4d86ac4508ef"
server_key_version: "encryptionkey@0"
I20260809 15:40:18.255801 28018 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskm314YR/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786289732823074-3431-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-taskm314YR/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786289732823074-3431-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-taskm314YR/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786289732823074-3431-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 15:40:18.272284 28018 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 15:40:18.275493 28018 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 15:40:18.275695 28018 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 15:40:18.282542 28018 rpc_server.cc:307] RPC server started. Bound to: 127.3.89.253:46795
I20260809 15:40:18.282833 28088 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.3.89.253:46795 every 8 connection(s)
I20260809 15:40:18.283567 28018 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskm314YR/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786289732823074-3431-0/minicluster-data/master-1/data/info.pb
I20260809 15:40:18.286052 28089 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 15:40:18.289592 28089 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.3.89.254" port: 32885 } has no permanent_uuid. Determining permanent_uuid...
I20260809 15:40:18.292732 3431 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskm314YR/build/debug/bin/kudu as pid 28018
I20260809 15:40:18.292901 3431 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskm314YR/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786289732823074-3431-0/minicluster-data/master-1/wal/instance
I20260809 15:40:18.293179 3431 external_mini_cluster.cc:1428] Setting key ee6177156c66c690cc658f163ec1f8dd
Aug 09 15:40:18 dist-test-slave-2w3w krb5kdc[27921](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786290018, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.89.253@KRBTEST.COM for kudu/127.3.89.254@KRBTEST.COM
WARNING: no policy specified for kudu/127.3.89.252@KRBTEST.COM; defaulting to no policy
I20260809 15:40:18.304487 28012 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:40:18.291411 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34418 (local address 127.3.89.254:32885)
0809 15:40:18.291561 (+ 150us) server_negotiation.cc:206] Beginning negotiation
0809 15:40:18.291566 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 15:40:18.292320 (+ 754us) server_negotiation.cc:407] Connection header received
0809 15:40:18.293370 (+ 1050us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 15:40:18.293373 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 15:40:18.293436 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 15:40:18.293539 (+ 103us) server_negotiation.cc:226] Negotiated authn=SASL
0809 15:40:18.294810 (+ 1271us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:40:18.295613 (+ 803us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:40:18.297375 (+ 1762us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:40:18.297613 (+ 238us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:40:18.297787 (+ 174us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:40:18.300703 (+ 2916us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 15:40:18.300706 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 15:40:18.300708 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 15:40:18.300734 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0809 15:40:18.302320 (+ 1586us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 15:40:18.302848 (+ 528us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 15:40:18.302852 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 15:40:18.302854 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 15:40:18.302938 (+ 84us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 15:40:18.303327 (+ 389us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 15:40:18.303331 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 15:40:18.303333 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 15:40:18.303567 (+ 234us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 15:40:18.303677 (+ 110us) server_negotiation.cc:1035] Waiting for connection context
0809 15:40:18.304179 (+ 502us) server_negotiation.cc:299] Negotiation successful
0809 15:40:18.304309 (+ 130us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":62}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.3.89.252@KRBTEST.COM" created.
I20260809 15:40:18.305444 28092 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:40:18.291740 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.89.254:32885 (local address 127.0.0.1:34418)
0809 15:40:18.292175 (+ 435us) negotiation.cc:107] Waiting for socket to connect
0809 15:40:18.292216 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0809 15:40:18.293123 (+ 907us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 15:40:18.293686 (+ 563us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 15:40:18.293692 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 15:40:18.294164 (+ 472us) client_negotiation.cc:189] Negotiated authn=SASL
0809 15:40:18.294650 (+ 486us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:40:18.294663 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:40:18.295750 (+ 1087us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:40:18.295756 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:40:18.297220 (+ 1464us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:40:18.297236 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:40:18.297842 (+ 606us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:40:18.297849 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:40:18.298122 (+ 273us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:40:18.298758 (+ 636us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 15:40:18.298781 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0809 15:40:18.300525 (+ 1744us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 15:40:18.302453 (+ 1928us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 15:40:18.302459 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 15:40:18.302473 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0809 15:40:18.302735 (+ 262us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 15:40:18.303063 (+ 328us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 15:40:18.303066 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 15:40:18.303068 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 15:40:18.303208 (+ 140us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 15:40:18.303676 (+ 468us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 15:40:18.303682 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 15:40:18.303930 (+ 248us) client_negotiation.cc:769] Sending connection context
0809 15:40:18.304121 (+ 191us) client_negotiation.cc:240] Negotiation successful
0809 15:40:18.304491 (+ 370us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":266,"thread_start_us":104,"threads_started":1}
I20260809 15:40:18.306849 28089 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.3.89.253" port: 46795 } has no permanent_uuid. Determining permanent_uuid...
I20260809 15:40:18.312059 28009 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.3.89.253" port: 46795 } attempt: 3
Aug 09 15:40:18 dist-test-slave-2w3w krb5kdc[27921](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786290018, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.89.253@KRBTEST.COM for kudu/127.3.89.253@KRBTEST.COM
I20260809 15:40:18.317157 28092 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:40:18.307416 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.89.253:46795 (local address 127.0.0.1:41624)
0809 15:40:18.307515 (+ 99us) negotiation.cc:107] Waiting for socket to connect
0809 15:40:18.307529 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0809 15:40:18.307617 (+ 88us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 15:40:18.308218 (+ 601us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 15:40:18.308221 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 15:40:18.308417 (+ 196us) client_negotiation.cc:189] Negotiated authn=SASL
0809 15:40:18.308493 (+ 76us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:40:18.308498 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:40:18.309458 (+ 960us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:40:18.309464 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:40:18.309976 (+ 512us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:40:18.309984 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:40:18.310567 (+ 583us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:40:18.310571 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:40:18.310662 (+ 91us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:40:18.311225 (+ 563us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 15:40:18.311236 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0809 15:40:18.312546 (+ 1310us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 15:40:18.314985 (+ 2439us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 15:40:18.314991 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 15:40:18.314996 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0809 15:40:18.315182 (+ 186us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 15:40:18.315501 (+ 319us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 15:40:18.315504 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 15:40:18.315505 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0809 15:40:18.315580 (+ 75us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 15:40:18.316130 (+ 550us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 15:40:18.316136 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 15:40:18.316288 (+ 152us) client_negotiation.cc:769] Sending connection context
0809 15:40:18.316902 (+ 614us) client_negotiation.cc:240] Negotiation successful
0809 15:40:18.317032 (+ 130us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":30}
Aug 09 15:40:18 dist-test-slave-2w3w krb5kdc[27921](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786290018, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.89.254@KRBTEST.COM for kudu/127.3.89.253@KRBTEST.COM
I20260809 15:40:18.317754 28096 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:40:18.307535 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41624 (local address 127.3.89.253:46795)
0809 15:40:18.307894 (+ 359us) server_negotiation.cc:206] Beginning negotiation
0809 15:40:18.307908 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0809 15:40:18.307926 (+ 18us) server_negotiation.cc:407] Connection header received
0809 15:40:18.307957 (+ 31us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 15:40:18.307961 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 15:40:18.308099 (+ 138us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 15:40:18.308240 (+ 141us) server_negotiation.cc:226] Negotiated authn=SASL
0809 15:40:18.308602 (+ 362us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:40:18.309340 (+ 738us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:40:18.310088 (+ 748us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:40:18.310453 (+ 365us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:40:18.310568 (+ 115us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:40:18.312676 (+ 2108us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 15:40:18.312687 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 15:40:18.312704 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 15:40:18.312733 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0809 15:40:18.314865 (+ 2132us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 15:40:18.315290 (+ 425us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 15:40:18.315298 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 15:40:18.315302 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 15:40:18.315387 (+ 85us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 15:40:18.315676 (+ 289us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 15:40:18.315679 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 15:40:18.315681 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 15:40:18.316002 (+ 321us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 15:40:18.316146 (+ 144us) server_negotiation.cc:1035] Waiting for connection context
0809 15:40:18.317090 (+ 944us) server_negotiation.cc:299] Negotiation successful
0809 15:40:18.317257 (+ 167us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":274,"thread_start_us":170,"threads_started":1}
I20260809 15:40:18.319674 28089 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.3.89.252" port: 39119 } has no permanent_uuid. Determining permanent_uuid...
I20260809 15:40:18.320744 28092 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:40:18.320217 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.89.252:39119 (local address 127.0.0.1:55050)
0809 15:40:18.320584 (+ 367us) negotiation.cc:107] Waiting for socket to connect
0809 15:40:18.320669 (+ 85us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.89.252:39119: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":267}
W20260809 15:40:18.321038 28037 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.3.89.252:39119: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.3.89.252 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786289732823074-3431-0/krb5kdc/kudu_127.3.89.252.keytab.
Entry for principal kudu/127.3.89.252 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786289732823074-3431-0/krb5kdc/kudu_127.3.89.252.keytab.
I20260809 15:40:18.321334 28014 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:40:18.312490 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.89.253:46795 (local address 127.0.0.1:41634)
0809 15:40:18.312677 (+ 187us) negotiation.cc:107] Waiting for socket to connect
0809 15:40:18.312692 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0809 15:40:18.312762 (+ 70us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 15:40:18.313205 (+ 443us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 15:40:18.313211 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 15:40:18.313373 (+ 162us) client_negotiation.cc:189] Negotiated authn=SASL
0809 15:40:18.313460 (+ 87us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:40:18.313466 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:40:18.314496 (+ 1030us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:40:18.314502 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:40:18.315151 (+ 649us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:40:18.315161 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:40:18.315706 (+ 545us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:40:18.315710 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:40:18.315826 (+ 116us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:40:18.316354 (+ 528us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 15:40:18.316365 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0809 15:40:18.317694 (+ 1329us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 15:40:18.319744 (+ 2050us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 15:40:18.319747 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 15:40:18.319749 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 15:40:18.320006 (+ 257us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 15:40:18.320296 (+ 290us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 15:40:18.320299 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 15:40:18.320301 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 15:40:18.320396 (+ 95us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 15:40:18.320791 (+ 395us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 15:40:18.320795 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 15:40:18.320904 (+ 109us) client_negotiation.cc:769] Sending connection context
0809 15:40:18.321032 (+ 128us) client_negotiation.cc:240] Negotiation successful
0809 15:40:18.321173 (+ 141us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":111}
I20260809 15:40:18.321349 28097 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:40:18.312576 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41634 (local address 127.3.89.253:46795)
0809 15:40:18.312938 (+ 362us) server_negotiation.cc:206] Beginning negotiation
0809 15:40:18.312947 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0809 15:40:18.312962 (+ 15us) server_negotiation.cc:407] Connection header received
0809 15:40:18.313002 (+ 40us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 15:40:18.313005 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 15:40:18.313058 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 15:40:18.313184 (+ 126us) server_negotiation.cc:226] Negotiated authn=SASL
0809 15:40:18.313585 (+ 401us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:40:18.314350 (+ 765us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:40:18.315275 (+ 925us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:40:18.315577 (+ 302us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:40:18.315689 (+ 112us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:40:18.317849 (+ 2160us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 15:40:18.317852 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 15:40:18.317855 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 15:40:18.317879 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0809 15:40:18.319550 (+ 1671us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 15:40:18.320118 (+ 568us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 15:40:18.320121 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 15:40:18.320122 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 15:40:18.320197 (+ 75us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 15:40:18.320504 (+ 307us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 15:40:18.320507 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 15:40:18.320509 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 15:40:18.320683 (+ 174us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 15:40:18.320762 (+ 79us) server_negotiation.cc:1035] Waiting for connection context
0809 15:40:18.321087 (+ 325us) server_negotiation.cc:299] Negotiation successful
0809 15:40:18.321194 (+ 107us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":283,"thread_start_us":115,"threads_started":1}
I20260809 15:40:18.322198 28092 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:40:18.321975 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.89.252:39119 (local address 127.0.0.1:55056)
0809 15:40:18.322080 (+ 105us) negotiation.cc:107] Waiting for socket to connect
0809 15:40:18.322138 (+ 58us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.89.252:39119: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":43}
I20260809 15:40:18.322227 3431 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskm314YR/build/debug/bin/kudu
/tmp/dist-test-taskm314YR/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786289732823074-3431-0/minicluster-data/master-2/wal
--fs_data_dirs=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786289732823074-3431-0/minicluster-data/master-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786289732823074-3431-0/minicluster-data/master-2/logs
--server_dump_info_path=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786289732823074-3431-0/minicluster-data/master-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.3.89.252:39119
--webserver_interface=127.3.89.252
--webserver_port=0
--keytab_file=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786289732823074-3431-0/krb5kdc/kudu_127.3.89.252.keytab
--principal=kudu/127.3.89.252
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.3.89.212:43203
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.3.89.254:32885,127.3.89.253:46795,127.3.89.252:39119
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=required with env {KRB5CCNAME=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786289732823074-3431-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786289732823074-3431-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786289732823074-3431-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260809 15:40:18.322315 28009 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.3.89.252" port: 39119 } has no permanent_uuid. Determining permanent_uuid...
W20260809 15:40:18.322683 28089 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.3.89.252:39119: Network error: Client connection negotiation failed: client connection to 127.3.89.252:39119: connect: Connection refused (error 111)
I20260809 15:40:18.323055 28014 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:40:18.322763 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.89.252:39119 (local address 127.0.0.1:55072)
0809 15:40:18.322886 (+ 123us) negotiation.cc:107] Waiting for socket to connect
0809 15:40:18.322934 (+ 48us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.89.252:39119: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":57}
I20260809 15:40:18.324301 28014 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:40:18.323978 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.89.252:39119 (local address 127.0.0.1:55086)
0809 15:40:18.324164 (+ 186us) negotiation.cc:107] Waiting for socket to connect
0809 15:40:18.324226 (+ 62us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.89.252:39119: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":102}
W20260809 15:40:18.324466 28009 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.3.89.252:39119: Network error: Client connection negotiation failed: client connection to 127.3.89.252:39119: connect: Connection refused (error 111)
I20260809 15:40:18.355624 28009 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.3.89.252" port: 39119 } attempt: 1
I20260809 15:40:18.356487 28014 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:40:18.356192 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.89.252:39119 (local address 127.0.0.1:34674)
0809 15:40:18.356345 (+ 153us) negotiation.cc:107] Waiting for socket to connect
0809 15:40:18.356408 (+ 63us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.89.252:39119: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":59}
I20260809 15:40:18.357215 28014 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:40:18.356994 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.89.252:39119 (local address 127.0.0.1:34678)
0809 15:40:18.357104 (+ 110us) negotiation.cc:107] Waiting for socket to connect
0809 15:40:18.357157 (+ 53us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.89.252:39119: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":44}
W20260809 15:40:18.357349 28009 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.3.89.252:39119: Network error: Client connection negotiation failed: client connection to 127.3.89.252:39119: connect: Connection refused (error 111)
I20260809 15:40:18.371965 28089 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.3.89.252" port: 39119 } attempt: 1
I20260809 15:40:18.372822 28092 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:40:18.372536 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.89.252:39119 (local address 127.0.0.1:34684)
0809 15:40:18.372688 (+ 152us) negotiation.cc:107] Waiting for socket to connect
0809 15:40:18.372749 (+ 61us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.89.252:39119: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":61}
I20260809 15:40:18.373582 28092 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:40:18.373353 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.89.252:39119 (local address 127.0.0.1:34700)
0809 15:40:18.373485 (+ 132us) negotiation.cc:107] Waiting for socket to connect
0809 15:40:18.373530 (+ 45us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.89.252:39119: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":43}
W20260809 15:40:18.373724 28089 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.3.89.252:39119: Network error: Client connection negotiation failed: client connection to 127.3.89.252:39119: connect: Connection refused (error 111)
I20260809 15:40:18.432513 28009 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.3.89.252" port: 39119 } attempt: 2
I20260809 15:40:18.433384 28014 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:40:18.433099 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.89.252:39119 (local address 127.0.0.1:34712)
0809 15:40:18.433245 (+ 146us) negotiation.cc:107] Waiting for socket to connect
0809 15:40:18.433308 (+ 63us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.89.252:39119: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":52}
I20260809 15:40:18.434134 28014 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:40:18.433889 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.89.252:39119 (local address 127.0.0.1:34720)
0809 15:40:18.434033 (+ 144us) negotiation.cc:107] Waiting for socket to connect
0809 15:40:18.434077 (+ 44us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.89.252:39119: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":45}
W20260809 15:40:18.434321 28009 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.3.89.252:39119: Network error: Client connection negotiation failed: client connection to 127.3.89.252:39119: connect: Connection refused (error 111)
W20260809 15:40:18.434333 28099 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260809 15:40:18.434643 28099 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 15:40:18.434676 28099 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 15:40:18.434732 28099 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 15:40:18.438747 28099 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 15:40:18.438804 28099 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 15:40:18.438824 28099 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 15:40:18.438841 28099 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 15:40:18.438864 28099 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 15:40:18.441895 28089 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.3.89.252" port: 39119 } attempt: 2
I20260809 15:40:18.442623 28092 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:40:18.442340 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.89.252:39119 (local address 127.0.0.1:34724)
0809 15:40:18.442492 (+ 152us) negotiation.cc:107] Waiting for socket to connect
0809 15:40:18.442553 (+ 61us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.89.252:39119: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":59}
I20260809 15:40:18.443382 28092 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:40:18.443186 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.89.252:39119 (local address 127.0.0.1:34730)
0809 15:40:18.443282 (+ 96us) negotiation.cc:107] Waiting for socket to connect
0809 15:40:18.443333 (+ 51us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.3.89.252:39119: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":40}
W20260809 15:40:18.443584 28089 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.3.89.252:39119: Network error: Client connection negotiation failed: client connection to 127.3.89.252:39119: connect: Connection refused (error 111)
I20260809 15:40:18.444499 28099 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.3.89.212:43203
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786289732823074-3431-0/minicluster-data/master-2/data
--fs_wal_dir=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786289732823074-3431-0/minicluster-data/master-2/wal
--ipki_ca_key_size=768
--master_addresses=127.3.89.254:32885,127.3.89.253:46795,127.3.89.252:39119
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786289732823074-3431-0/krb5kdc/kudu_127.3.89.252.keytab
--principal=kudu/127.3.89.252
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.3.89.252:39119
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786289732823074-3431-0/minicluster-data/master-2/data/info.pb
--webserver_interface=127.3.89.252
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.28099
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786289732823074-3431-0/minicluster-data/master-2/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 09 Aug 2026 14:46:55 UTC on ccycloud.quasar-yswkmf.root.comops.site
build id 4576540
I20260809 15:40:18.445544 28099 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 15:40:18.446539 28099 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 15:40:18.455075 28105 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 15:40:18.455156 28104 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 15:40:18.455082 28107 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 15:40:18.455220 28099 server_base.cc:1034] running on GCE node
I20260809 15:40:18.455886 28099 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 15:40:18.456895 28099 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 15:40:18.458060 28099 hybrid_clock.cc:648] HybridClock initialized: now 1786290018458031 us; error 38 us; skew 500 ppm
Aug 09 15:40:18 dist-test-slave-2w3w krb5kdc[27921](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786290018, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.89.252@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 15:40:18.466820 28099 init.cc:377] Logged in from keytab as kudu/127.3.89.252@KRBTEST.COM (short username kudu)
I20260809 15:40:18.467721 28099 webserver.cc:460] Webserver started at http://127.3.89.252:37071/ using document root <none> and password file <none>
I20260809 15:40:18.468302 28099 fs_manager.cc:359] Metadata directory not provided
I20260809 15:40:18.468366 28099 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 15:40:18.468597 28099 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 15:40:18.470660 28099 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskm314YR/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786289732823074-3431-0/minicluster-data/master-2/data/instance:
uuid: "cbe6b1b7953e4d8ca41ba1fb96553973"
format_stamp: "Formatted at 2026-08-09 15:40:18 on dist-test-slave-2w3w"
server_key: "b7c763f6996c7b6e8948710b6f6bcf08"
server_key_iv: "9e618fa307c67f1135a6aad6618683e5"
server_key_version: "encryptionkey@0"
I20260809 15:40:18.471262 28099 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskm314YR/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786289732823074-3431-0/minicluster-data/master-2/wal/instance:
uuid: "cbe6b1b7953e4d8ca41ba1fb96553973"
format_stamp: "Formatted at 2026-08-09 15:40:18 on dist-test-slave-2w3w"
server_key: "b7c763f6996c7b6e8948710b6f6bcf08"
server_key_iv: "9e618fa307c67f1135a6aad6618683e5"
server_key_version: "encryptionkey@0"
I20260809 15:40:18.474844 28099 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.000s sys 0.003s
I20260809 15:40:18.477147 28115 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 15:40:18.478354 28099 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260809 15:40:18.478497 28099 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskm314YR/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786289732823074-3431-0/minicluster-data/master-2/data,/tmp/dist-test-taskm314YR/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786289732823074-3431-0/minicluster-data/master-2/wal
uuid: "cbe6b1b7953e4d8ca41ba1fb96553973"
format_stamp: "Formatted at 2026-08-09 15:40:18 on dist-test-slave-2w3w"
server_key: "b7c763f6996c7b6e8948710b6f6bcf08"
server_key_iv: "9e618fa307c67f1135a6aad6618683e5"
server_key_version: "encryptionkey@0"
I20260809 15:40:18.478607 28099 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskm314YR/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786289732823074-3431-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-taskm314YR/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786289732823074-3431-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-taskm314YR/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786289732823074-3431-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 15:40:18.510509 28099 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 15:40:18.513916 28099 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 15:40:18.514134 28099 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 15:40:18.521121 28099 rpc_server.cc:307] RPC server started. Bound to: 127.3.89.252:39119
I20260809 15:40:18.521409 28170 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.3.89.252:39119 every 8 connection(s)
I20260809 15:40:18.522102 28099 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskm314YR/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786289732823074-3431-0/minicluster-data/master-2/data/info.pb
I20260809 15:40:18.524839 28171 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 15:40:18.528775 28171 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.3.89.254" port: 32885 } has no permanent_uuid. Determining permanent_uuid...
I20260809 15:40:18.530143 3431 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskm314YR/build/debug/bin/kudu as pid 28099
I20260809 15:40:18.530308 3431 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskm314YR/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786289732823074-3431-0/minicluster-data/master-2/wal/instance
I20260809 15:40:18.530616 3431 external_mini_cluster.cc:1428] Setting key 9ded49dcb3465144a3625b214541e522
I20260809 15:40:18.533520 28009 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.3.89.252" port: 39119 } attempt: 3
I20260809 15:40:18.534756 28089 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.3.89.252" port: 39119 } attempt: 3
Aug 09 15:40:18 dist-test-slave-2w3w krb5kdc[27921](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786290018, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.89.252@KRBTEST.COM for kudu/127.3.89.254@KRBTEST.COM
Aug 09 15:40:18 dist-test-slave-2w3w krb5kdc[27921](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786290017, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.3.89.252@KRBTEST.COM
Aug 09 15:40:18 dist-test-slave-2w3w krb5kdc[27921](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786290018, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.89.254@KRBTEST.COM for kudu/127.3.89.252@KRBTEST.COM
Aug 09 15:40:18 dist-test-slave-2w3w krb5kdc[27921](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786290018, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.89.253@KRBTEST.COM for kudu/127.3.89.252@KRBTEST.COM
I20260809 15:40:18.545323 28012 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:40:18.530824 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36148 (local address 127.3.89.254:32885)
0809 15:40:18.531028 (+ 204us) server_negotiation.cc:206] Beginning negotiation
0809 15:40:18.531032 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 15:40:18.532024 (+ 992us) server_negotiation.cc:407] Connection header received
0809 15:40:18.532889 (+ 865us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 15:40:18.532893 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 15:40:18.532941 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 15:40:18.533033 (+ 92us) server_negotiation.cc:226] Negotiated authn=SASL
0809 15:40:18.534165 (+ 1132us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:40:18.534996 (+ 831us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:40:18.536437 (+ 1441us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:40:18.536789 (+ 352us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:40:18.536877 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:40:18.540373 (+ 3496us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 15:40:18.540376 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 15:40:18.540381 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 15:40:18.540408 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0809 15:40:18.542522 (+ 2114us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 15:40:18.543326 (+ 804us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 15:40:18.543332 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 15:40:18.543335 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 15:40:18.543437 (+ 102us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 15:40:18.543882 (+ 445us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 15:40:18.543886 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 15:40:18.543889 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 15:40:18.544214 (+ 325us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 15:40:18.544309 (+ 95us) server_negotiation.cc:1035] Waiting for connection context
0809 15:40:18.544944 (+ 635us) server_negotiation.cc:299] Negotiation successful
0809 15:40:18.545091 (+ 147us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":3,"server-negotiator.queue_time_us":64}
I20260809 15:40:18.546128 28014 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:40:18.533958 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.89.252:39119 (local address 127.0.0.1:34748)
0809 15:40:18.534147 (+ 189us) negotiation.cc:107] Waiting for socket to connect
0809 15:40:18.534164 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0809 15:40:18.534244 (+ 80us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 15:40:18.534536 (+ 292us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 15:40:18.534539 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 15:40:18.534747 (+ 208us) client_negotiation.cc:189] Negotiated authn=SASL
0809 15:40:18.534862 (+ 115us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:40:18.534872 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:40:18.535923 (+ 1051us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:40:18.535927 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:40:18.536984 (+ 1057us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:40:18.536994 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:40:18.538346 (+ 1352us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:40:18.538353 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:40:18.538449 (+ 96us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:40:18.538985 (+ 536us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 15:40:18.539008 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0809 15:40:18.540973 (+ 1965us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 15:40:18.544248 (+ 3275us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 15:40:18.544251 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 15:40:18.544253 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 15:40:18.544531 (+ 278us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 15:40:18.544887 (+ 356us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 15:40:18.544892 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 15:40:18.544893 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0809 15:40:18.544988 (+ 95us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 15:40:18.545650 (+ 662us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 15:40:18.545653 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 15:40:18.545746 (+ 93us) client_negotiation.cc:769] Sending connection context
0809 15:40:18.545842 (+ 96us) client_negotiation.cc:240] Negotiation successful
0809 15:40:18.545970 (+ 128us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":114,"mutex_wait_us":244}
I20260809 15:40:18.546128 28092 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:40:18.535200 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.89.252:39119 (local address 127.0.0.1:34756)
0809 15:40:18.535330 (+ 130us) negotiation.cc:107] Waiting for socket to connect
0809 15:40:18.535345 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0809 15:40:18.535423 (+ 78us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 15:40:18.535945 (+ 522us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 15:40:18.535949 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 15:40:18.536145 (+ 196us) client_negotiation.cc:189] Negotiated authn=SASL
0809 15:40:18.536252 (+ 107us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:40:18.536259 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:40:18.537334 (+ 1075us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:40:18.537338 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:40:18.538116 (+ 778us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:40:18.538129 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:40:18.538732 (+ 603us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:40:18.538737 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:40:18.538856 (+ 119us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:40:18.539282 (+ 426us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 15:40:18.539297 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0809 15:40:18.541598 (+ 2301us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 15:40:18.544387 (+ 2789us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 15:40:18.544391 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 15:40:18.544393 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 15:40:18.544644 (+ 251us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 15:40:18.544952 (+ 308us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 15:40:18.544955 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 15:40:18.544957 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 15:40:18.545047 (+ 90us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 15:40:18.545650 (+ 603us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 15:40:18.545654 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 15:40:18.545746 (+ 92us) client_negotiation.cc:769] Sending connection context
0809 15:40:18.545842 (+ 96us) client_negotiation.cc:240] Negotiation successful
0809 15:40:18.545970 (+ 128us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":43}
I20260809 15:40:18.547891 28174 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:40:18.531342 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.89.254:32885 (local address 127.0.0.1:36148)
0809 15:40:18.531846 (+ 504us) negotiation.cc:107] Waiting for socket to connect
0809 15:40:18.531897 (+ 51us) client_negotiation.cc:174] Beginning negotiation
0809 15:40:18.532689 (+ 792us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 15:40:18.533127 (+ 438us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 15:40:18.533133 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 15:40:18.533601 (+ 468us) client_negotiation.cc:189] Negotiated authn=SASL
0809 15:40:18.533992 (+ 391us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:40:18.534015 (+ 23us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:40:18.535137 (+ 1122us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:40:18.535140 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:40:18.536303 (+ 1163us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:40:18.536313 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:40:18.537082 (+ 769us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:40:18.537088 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:40:18.537265 (+ 177us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:40:18.537973 (+ 708us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 15:40:18.537992 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0809 15:40:18.540236 (+ 2244us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 15:40:18.542657 (+ 2421us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 15:40:18.542668 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 15:40:18.542681 (+ 13us) client_negotiation.cc:760] calling sasl_client_step()
0809 15:40:18.543188 (+ 507us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 15:40:18.543569 (+ 381us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 15:40:18.543573 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 15:40:18.543577 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0809 15:40:18.543768 (+ 191us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 15:40:18.544302 (+ 534us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 15:40:18.544308 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 15:40:18.544620 (+ 312us) client_negotiation.cc:769] Sending connection context
0809 15:40:18.545905 (+ 1285us) client_negotiation.cc:240] Negotiation successful
0809 15:40:18.546793 (+ 888us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":310,"spinlock_wait_cycles":2816,"thread_start_us":150,"threads_started":1}
I20260809 15:40:18.547894 28177 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:40:18.533987 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34748 (local address 127.3.89.252:39119)
0809 15:40:18.534266 (+ 279us) server_negotiation.cc:206] Beginning negotiation
0809 15:40:18.534276 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0809 15:40:18.534289 (+ 13us) server_negotiation.cc:407] Connection header received
0809 15:40:18.534370 (+ 81us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 15:40:18.534375 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 15:40:18.534414 (+ 39us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 15:40:18.534520 (+ 106us) server_negotiation.cc:226] Negotiated authn=SASL
0809 15:40:18.535023 (+ 503us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:40:18.535789 (+ 766us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:40:18.537099 (+ 1310us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:40:18.538201 (+ 1102us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:40:18.539737 (+ 1536us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:40:18.541116 (+ 1379us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 15:40:18.541120 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 15:40:18.541124 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 15:40:18.541153 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0809 15:40:18.544129 (+ 2976us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 15:40:18.544662 (+ 533us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 15:40:18.544671 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 15:40:18.544674 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 15:40:18.544775 (+ 101us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 15:40:18.545108 (+ 333us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 15:40:18.545114 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 15:40:18.545119 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0809 15:40:18.545521 (+ 402us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 15:40:18.546426 (+ 905us) server_negotiation.cc:1035] Waiting for connection context
0809 15:40:18.546557 (+ 131us) server_negotiation.cc:299] Negotiation successful
0809 15:40:18.547036 (+ 479us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":441,"server-negotiator.queue_time_us":223,"spinlock_wait_cycles":8704,"thread_start_us":99,"threads_started":1}
I20260809 15:40:18.547891 28176 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:40:18.531357 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34744 (local address 127.3.89.252:39119)
0809 15:40:18.531936 (+ 579us) server_negotiation.cc:206] Beginning negotiation
0809 15:40:18.531946 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0809 15:40:18.532009 (+ 63us) server_negotiation.cc:407] Connection header received
0809 15:40:18.532673 (+ 664us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 15:40:18.532689 (+ 16us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 15:40:18.532930 (+ 241us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 15:40:18.533097 (+ 167us) server_negotiation.cc:226] Negotiated authn=SASL
0809 15:40:18.534071 (+ 974us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:40:18.535664 (+ 1593us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:40:18.536862 (+ 1198us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:40:18.537428 (+ 566us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:40:18.537516 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:40:18.540632 (+ 3116us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 15:40:18.540639 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 15:40:18.540650 (+ 11us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 15:40:18.540677 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0809 15:40:18.544563 (+ 3886us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 15:40:18.545066 (+ 503us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 15:40:18.545070 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 15:40:18.545071 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 15:40:18.545148 (+ 77us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 15:40:18.545456 (+ 308us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 15:40:18.545472 (+ 16us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 15:40:18.545473 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 15:40:18.545704 (+ 231us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 15:40:18.545879 (+ 175us) server_negotiation.cc:1035] Waiting for connection context
0809 15:40:18.546550 (+ 671us) server_negotiation.cc:299] Negotiation successful
0809 15:40:18.546849 (+ 299us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":146,"server-negotiator.queue_time_us":468,"spinlock_wait_cycles":59520,"thread_start_us":178,"threads_started":1}
I20260809 15:40:18.547894 28178 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:40:18.535428 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34756 (local address 127.3.89.252:39119)
0809 15:40:18.535701 (+ 273us) server_negotiation.cc:206] Beginning negotiation
0809 15:40:18.535709 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0809 15:40:18.535721 (+ 12us) server_negotiation.cc:407] Connection header received
0809 15:40:18.535751 (+ 30us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 15:40:18.535754 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 15:40:18.535804 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 15:40:18.536366 (+ 562us) server_negotiation.cc:226] Negotiated authn=SASL
0809 15:40:18.536457 (+ 91us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:40:18.537201 (+ 744us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:40:18.538260 (+ 1059us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:40:18.538587 (+ 327us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:40:18.539088 (+ 501us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:40:18.541727 (+ 2639us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 15:40:18.541731 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 15:40:18.541735 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 15:40:18.541771 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0809 15:40:18.544258 (+ 2487us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 15:40:18.544756 (+ 498us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 15:40:18.544759 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 15:40:18.544760 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 15:40:18.544848 (+ 88us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 15:40:18.545157 (+ 309us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 15:40:18.545163 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 15:40:18.545166 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 15:40:18.545521 (+ 355us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 15:40:18.546426 (+ 905us) server_negotiation.cc:1035] Waiting for connection context
0809 15:40:18.546557 (+ 131us) server_negotiation.cc:299] Negotiation successful
0809 15:40:18.546954 (+ 397us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":250,"server-negotiator.queue_time_us":219,"spinlock_wait_cycles":67200,"thread_start_us":68,"threads_started":1}
I20260809 15:40:18.549211 28171 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.3.89.253" port: 46795 } has no permanent_uuid. Determining permanent_uuid...
I20260809 15:40:18.553164 28009 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8ba587d8bdca4e27ac7a45bbf5f8a260: Bootstrap starting.
I20260809 15:40:18.553164 28089 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c064bc9a409c478cb810307783d32488: Bootstrap starting.
Aug 09 15:40:18 dist-test-slave-2w3w krb5kdc[27921](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786290018, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.89.252@KRBTEST.COM for kudu/127.3.89.253@KRBTEST.COM
I20260809 15:40:18.556078 28089 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P c064bc9a409c478cb810307783d32488: Neither blocks nor log segments found. Creating new log.
I20260809 15:40:18.556190 28009 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 8ba587d8bdca4e27ac7a45bbf5f8a260: Neither blocks nor log segments found. Creating new log.
Aug 09 15:40:18 dist-test-slave-2w3w krb5kdc[27921](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786290017, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.3.89.254@KRBTEST.COM
I20260809 15:40:18.557017 28089 log.cc:824] T 00000000000000000000000000000000 P c064bc9a409c478cb810307783d32488: Log is configured to *not* fsync() on all Append() calls
I20260809 15:40:18.557039 28009 log.cc:824] T 00000000000000000000000000000000 P 8ba587d8bdca4e27ac7a45bbf5f8a260: Log is configured to *not* fsync() on all Append() calls
I20260809 15:40:18.559485 28089 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c064bc9a409c478cb810307783d32488: No bootstrap required, opened a new log
I20260809 15:40:18.559859 28174 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:40:18.549699 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.89.253:46795 (local address 127.0.0.1:35750)
0809 15:40:18.549898 (+ 199us) negotiation.cc:107] Waiting for socket to connect
0809 15:40:18.549916 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0809 15:40:18.549998 (+ 82us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 15:40:18.550220 (+ 222us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 15:40:18.550226 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 15:40:18.550486 (+ 260us) client_negotiation.cc:189] Negotiated authn=SASL
0809 15:40:18.550628 (+ 142us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:40:18.550636 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:40:18.551685 (+ 1049us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:40:18.551688 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:40:18.552451 (+ 763us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:40:18.552461 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:40:18.553135 (+ 674us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:40:18.553138 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:40:18.553257 (+ 119us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:40:18.553740 (+ 483us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 15:40:18.553754 (+ 14us) client_negotiation.cc:656] calling sasl_client_start()
0809 15:40:18.555216 (+ 1462us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 15:40:18.557630 (+ 2414us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 15:40:18.557633 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 15:40:18.557635 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 15:40:18.557893 (+ 258us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 15:40:18.558271 (+ 378us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 15:40:18.558277 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 15:40:18.558282 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0809 15:40:18.558376 (+ 94us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 15:40:18.558926 (+ 550us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 15:40:18.558929 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 15:40:18.559049 (+ 120us) client_negotiation.cc:769] Sending connection context
0809 15:40:18.559576 (+ 527us) client_negotiation.cc:240] Negotiation successful
0809 15:40:18.559709 (+ 133us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":99}
I20260809 15:40:18.559911 28009 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8ba587d8bdca4e27ac7a45bbf5f8a260: No bootstrap required, opened a new log
I20260809 15:40:18.560566 28012 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:40:18.550430 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36162 (local address 127.3.89.254:32885)
0809 15:40:18.550535 (+ 105us) server_negotiation.cc:206] Beginning negotiation
0809 15:40:18.550541 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 15:40:18.550599 (+ 58us) server_negotiation.cc:407] Connection header received
0809 15:40:18.550768 (+ 169us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 15:40:18.550771 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 15:40:18.550812 (+ 41us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 15:40:18.550858 (+ 46us) server_negotiation.cc:226] Negotiated authn=SASL
0809 15:40:18.551758 (+ 900us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:40:18.552459 (+ 701us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:40:18.553525 (+ 1066us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:40:18.553848 (+ 323us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:40:18.553950 (+ 102us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:40:18.556832 (+ 2882us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 15:40:18.556837 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 15:40:18.556842 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 15:40:18.556868 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0809 15:40:18.558563 (+ 1695us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 15:40:18.559092 (+ 529us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 15:40:18.559096 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 15:40:18.559097 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 15:40:18.559190 (+ 93us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 15:40:18.559534 (+ 344us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 15:40:18.559539 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 15:40:18.559541 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 15:40:18.559767 (+ 226us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 15:40:18.559936 (+ 169us) server_negotiation.cc:1035] Waiting for connection context
0809 15:40:18.560320 (+ 384us) server_negotiation.cc:299] Negotiation successful
0809 15:40:18.560436 (+ 116us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
I20260809 15:40:18.563730 28009 raft_consensus.cc:348] T 00000000000000000000000000000000 P 8ba587d8bdca4e27ac7a45bbf5f8a260 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8ba587d8bdca4e27ac7a45bbf5f8a260" member_type: VOTER last_known_addr { host: "127.3.89.254" port: 32885 } } peers { permanent_uuid: "c064bc9a409c478cb810307783d32488" member_type: VOTER last_known_addr { host: "127.3.89.253" port: 46795 } } peers { permanent_uuid: "cbe6b1b7953e4d8ca41ba1fb96553973" member_type: VOTER last_known_addr { host: "127.3.89.252" port: 39119 } }
I20260809 15:40:18.563797 28089 raft_consensus.cc:348] T 00000000000000000000000000000000 P c064bc9a409c478cb810307783d32488 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8ba587d8bdca4e27ac7a45bbf5f8a260" member_type: VOTER last_known_addr { host: "127.3.89.254" port: 32885 } } peers { permanent_uuid: "c064bc9a409c478cb810307783d32488" member_type: VOTER last_known_addr { host: "127.3.89.253" port: 46795 } } peers { permanent_uuid: "cbe6b1b7953e4d8ca41ba1fb96553973" member_type: VOTER last_known_addr { host: "127.3.89.252" port: 39119 } }
I20260809 15:40:18.563933 28009 raft_consensus.cc:729] T 00000000000000000000000000000000 P 8ba587d8bdca4e27ac7a45bbf5f8a260 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8ba587d8bdca4e27ac7a45bbf5f8a260, State: Initialized, Role: FOLLOWER
I20260809 15:40:18.564000 28089 raft_consensus.cc:729] T 00000000000000000000000000000000 P c064bc9a409c478cb810307783d32488 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c064bc9a409c478cb810307783d32488, State: Initialized, Role: FOLLOWER
I20260809 15:40:18.564236 28009 consensus_queue.cc:260] T 00000000000000000000000000000000 P 8ba587d8bdca4e27ac7a45bbf5f8a260 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8ba587d8bdca4e27ac7a45bbf5f8a260" member_type: VOTER last_known_addr { host: "127.3.89.254" port: 32885 } } peers { permanent_uuid: "c064bc9a409c478cb810307783d32488" member_type: VOTER last_known_addr { host: "127.3.89.253" port: 46795 } } peers { permanent_uuid: "cbe6b1b7953e4d8ca41ba1fb96553973" member_type: VOTER last_known_addr { host: "127.3.89.252" port: 39119 } }
I20260809 15:40:18.564281 28089 consensus_queue.cc:260] T 00000000000000000000000000000000 P c064bc9a409c478cb810307783d32488 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8ba587d8bdca4e27ac7a45bbf5f8a260" member_type: VOTER last_known_addr { host: "127.3.89.254" port: 32885 } } peers { permanent_uuid: "c064bc9a409c478cb810307783d32488" member_type: VOTER last_known_addr { host: "127.3.89.253" port: 46795 } } peers { permanent_uuid: "cbe6b1b7953e4d8ca41ba1fb96553973" member_type: VOTER last_known_addr { host: "127.3.89.252" port: 39119 } }
I20260809 15:40:18.564750 28182 sys_catalog.cc:455] T 00000000000000000000000000000000 P c064bc9a409c478cb810307783d32488 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8ba587d8bdca4e27ac7a45bbf5f8a260" member_type: VOTER last_known_addr { host: "127.3.89.254" port: 32885 } } peers { permanent_uuid: "c064bc9a409c478cb810307783d32488" member_type: VOTER last_known_addr { host: "127.3.89.253" port: 46795 } } peers { permanent_uuid: "cbe6b1b7953e4d8ca41ba1fb96553973" member_type: VOTER last_known_addr { host: "127.3.89.252" port: 39119 } } }
I20260809 15:40:18.565754 28182 sys_catalog.cc:458] T 00000000000000000000000000000000 P c064bc9a409c478cb810307783d32488 [sys.catalog]: This master's current role is: FOLLOWER
I20260809 15:40:18.565622 28089 sys_catalog.cc:564] T 00000000000000000000000000000000 P c064bc9a409c478cb810307783d32488 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 15:40:18.565860 28009 sys_catalog.cc:564] T 00000000000000000000000000000000 P 8ba587d8bdca4e27ac7a45bbf5f8a260 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 15:40:18.559522 28097 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:40:18.549788 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35750 (local address 127.3.89.253:46795)
0809 15:40:18.549907 (+ 119us) server_negotiation.cc:206] Beginning negotiation
0809 15:40:18.549910 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0809 15:40:18.549963 (+ 53us) server_negotiation.cc:407] Connection header received
0809 15:40:18.550077 (+ 114us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 15:40:18.550079 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 15:40:18.550112 (+ 33us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 15:40:18.550175 (+ 63us) server_negotiation.cc:226] Negotiated authn=SASL
0809 15:40:18.550763 (+ 588us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:40:18.551517 (+ 754us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:40:18.552590 (+ 1073us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:40:18.552916 (+ 326us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:40:18.552992 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:40:18.555365 (+ 2373us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 15:40:18.555369 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 15:40:18.555371 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 15:40:18.555395 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0809 15:40:18.557396 (+ 2001us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 15:40:18.558038 (+ 642us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 15:40:18.558042 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 15:40:18.558045 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 15:40:18.558139 (+ 94us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 15:40:18.558496 (+ 357us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 15:40:18.558501 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 15:40:18.558503 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 15:40:18.558762 (+ 259us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 15:40:18.558837 (+ 75us) server_negotiation.cc:1035] Waiting for connection context
0809 15:40:18.559254 (+ 417us) server_negotiation.cc:299] Negotiation successful
0809 15:40:18.559373 (+ 119us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":44}
I20260809 15:40:18.566659 28171 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.3.89.252" port: 39119 } has no permanent_uuid. Determining permanent_uuid...
Aug 09 15:40:18 dist-test-slave-2w3w krb5kdc[27921](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786290017, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.3.89.253@KRBTEST.COM
I20260809 15:40:18.568217 28181 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8ba587d8bdca4e27ac7a45bbf5f8a260 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8ba587d8bdca4e27ac7a45bbf5f8a260" member_type: VOTER last_known_addr { host: "127.3.89.254" port: 32885 } } peers { permanent_uuid: "c064bc9a409c478cb810307783d32488" member_type: VOTER last_known_addr { host: "127.3.89.253" port: 46795 } } peers { permanent_uuid: "cbe6b1b7953e4d8ca41ba1fb96553973" member_type: VOTER last_known_addr { host: "127.3.89.252" port: 39119 } } }
I20260809 15:40:18.568434 28181 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8ba587d8bdca4e27ac7a45bbf5f8a260 [sys.catalog]: This master's current role is: FOLLOWER
Aug 09 15:40:18 dist-test-slave-2w3w krb5kdc[27921](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786290018, etypes {rep=17 tkt=17 ses=17}, kudu/127.3.89.252@KRBTEST.COM for kudu/127.3.89.252@KRBTEST.COM
W20260809 15:40:18.573083 28200 catalog_manager.cc:1558] T 00000000000000000000000000000000 P c064bc9a409c478cb810307783d32488: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260809 15:40:18.573182 28200 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260809 15:40:18.570174 28018 master_runner.cc:186] Error getting master registration for 127.3.89.254:32885: OK, instance_id { permanent_uuid: "8ba587d8bdca4e27ac7a45bbf5f8a260" instance_seqno: 1786290018068260 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260809 15:40:18.576299 28174 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:40:18.567235 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.3.89.252:39119 (local address 127.0.0.1:34766)
0809 15:40:18.567463 (+ 228us) negotiation.cc:107] Waiting for socket to connect
0809 15:40:18.567478 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0809 15:40:18.567558 (+ 80us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 15:40:18.568041 (+ 483us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 15:40:18.568043 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 15:40:18.568206 (+ 163us) client_negotiation.cc:189] Negotiated authn=SASL
0809 15:40:18.568319 (+ 113us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:40:18.568326 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:40:18.569443 (+ 1117us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:40:18.569447 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:40:18.570311 (+ 864us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 15:40:18.570320 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 15:40:18.570710 (+ 390us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 15:40:18.570713 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 15:40:18.570784 (+ 71us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:40:18.571226 (+ 442us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 15:40:18.571240 (+ 14us) client_negotiation.cc:656] calling sasl_client_start()
0809 15:40:18.572720 (+ 1480us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 15:40:18.574668 (+ 1948us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 15:40:18.574671 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 15:40:18.574673 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 15:40:18.574930 (+ 257us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 15:40:18.575377 (+ 447us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 15:40:18.575380 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 15:40:18.575382 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 15:40:18.575471 (+ 89us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 15:40:18.575880 (+ 409us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 15:40:18.575883 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 15:40:18.575961 (+ 78us) client_negotiation.cc:769] Sending connection context
0809 15:40:18.576037 (+ 76us) client_negotiation.cc:240] Negotiation successful
0809 15:40:18.576155 (+ 118us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":155}
I20260809 15:40:18.576901 28176 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:40:18.567693 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34766 (local address 127.3.89.252:39119)
0809 15:40:18.567828 (+ 135us) server_negotiation.cc:206] Beginning negotiation
0809 15:40:18.567832 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 15:40:18.567845 (+ 13us) server_negotiation.cc:407] Connection header received
0809 15:40:18.567882 (+ 37us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 15:40:18.567887 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 15:40:18.567929 (+ 42us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 15:40:18.568416 (+ 487us) server_negotiation.cc:226] Negotiated authn=SASL
0809 15:40:18.568523 (+ 107us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:40:18.569321 (+ 798us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:40:18.570412 (+ 1091us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:40:18.570607 (+ 195us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:40:18.570710 (+ 103us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:40:18.572857 (+ 2147us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 15:40:18.572860 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 15:40:18.572862 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 15:40:18.572888 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0809 15:40:18.574556 (+ 1668us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 15:40:18.575079 (+ 523us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 15:40:18.575081 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 15:40:18.575083 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 15:40:18.575158 (+ 75us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 15:40:18.575568 (+ 410us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 15:40:18.575570 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 15:40:18.575572 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 15:40:18.575761 (+ 189us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 15:40:18.576555 (+ 794us) server_negotiation.cc:1035] Waiting for connection context
0809 15:40:18.576624 (+ 69us) server_negotiation.cc:299] Negotiation successful
0809 15:40:18.576726 (+ 102us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":64}
W20260809 15:40:18.577368 28204 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 8ba587d8bdca4e27ac7a45bbf5f8a260: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260809 15:40:18.577474 28204 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260809 15:40:18.574175 28096 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:40:18.562171 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35752 (local address 127.3.89.253:46795)
0809 15:40:18.562283 (+ 112us) server_negotiation.cc:206] Beginning negotiation
0809 15:40:18.562286 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0809 15:40:18.562297 (+ 11us) server_negotiation.cc:407] Connection header received
0809 15:40:18.562421 (+ 124us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 15:40:18.562424 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 15:40:18.562467 (+ 43us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 15:40:18.562516 (+ 49us) server_negotiation.cc:226] Negotiated authn=SASL
0809 15:40:18.563403 (+ 887us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:40:18.564119 (+ 716us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:40:18.565320 (+ 1201us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:40:18.565637 (+ 317us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:40:18.565720 (+ 83us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:40:18.568851 (+ 3131us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 15:40:18.568855 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 15:40:18.568858 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 15:40:18.568882 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0809 15:40:18.572006 (+ 3124us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 15:40:18.572546 (+ 540us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 15:40:18.572550 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 15:40:18.572552 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 15:40:18.572639 (+ 87us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 15:40:18.572973 (+ 334us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 15:40:18.572979 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 15:40:18.572982 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 15:40:18.573219 (+ 237us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 15:40:18.573429 (+ 210us) server_negotiation.cc:1035] Waiting for connection context
0809 15:40:18.573856 (+ 427us) server_negotiation.cc:299] Negotiation successful
0809 15:40:18.573989 (+ 133us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":45}
I20260809 15:40:18.579756 28171 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P cbe6b1b7953e4d8ca41ba1fb96553973: Bootstrap starting.
I20260809 15:40:18.581898 28171 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P cbe6b1b7953e4d8ca41ba1fb96553973: Neither blocks nor log segments found. Creating new log.
I20260809 15:40:18.578408 27934 master_runner.cc:186] Error getting master registration for 127.3.89.252:39119: OK, instance_id { permanent_uuid: "cbe6b1b7953e4d8ca41ba1fb96553973" instance_seqno: 1786290018519051 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260809 15:40:18.582417 27934 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260809 15:40:18.582813 28171 log.cc:824] T 00000000000000000000000000000000 P cbe6b1b7953e4d8ca41ba1fb96553973: Log is configured to *not* fsync() on all Append() calls
I20260809 15:40:18.582872 28018 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260809 15:40:18.584434 3431 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260809 15:40:18.585525 27980 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "00000000000000000000000000000000"
dest_uuid: "8ba587d8bdca4e27ac7a45bbf5f8a260"
from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:36162
I20260809 15:40:18.585734 27980 raft_consensus.cc:482] T 00000000000000000000000000000000 P 8ba587d8bdca4e27ac7a45bbf5f8a260 [term 0 FOLLOWER]: Starting forced leader election (received explicit request)
I20260809 15:40:18.585872 27980 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 8ba587d8bdca4e27ac7a45bbf5f8a260 [term 0 FOLLOWER]: Advancing to term 1
I20260809 15:40:18.586949 27980 raft_consensus.cc:504] T 00000000000000000000000000000000 P 8ba587d8bdca4e27ac7a45bbf5f8a260 [term 1 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8ba587d8bdca4e27ac7a45bbf5f8a260" member_type: VOTER last_known_addr { host: "127.3.89.254" port: 32885 } } peers { permanent_uuid: "c064bc9a409c478cb810307783d32488" member_type: VOTER last_known_addr { host: "127.3.89.253" port: 46795 } } peers { permanent_uuid: "cbe6b1b7953e4d8ca41ba1fb96553973" member_type: VOTER last_known_addr { host: "127.3.89.252" port: 39119 } }
I20260809 15:40:18.587916 27980 leader_election.cc:288] T 00000000000000000000000000000000 P 8ba587d8bdca4e27ac7a45bbf5f8a260 [CANDIDATE]: Term 1 election: Requested vote from peers c064bc9a409c478cb810307783d32488 (127.3.89.253:46795), cbe6b1b7953e4d8ca41ba1fb96553973 (127.3.89.252:39119)
I20260809 15:40:18.588310 28062 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "8ba587d8bdca4e27ac7a45bbf5f8a260" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: true dest_uuid: "c064bc9a409c478cb810307783d32488"
I20260809 15:40:18.588541 28062 raft_consensus.cc:3037] T 00000000000000000000000000000000 P c064bc9a409c478cb810307783d32488 [term 0 FOLLOWER]: Advancing to term 1
I20260809 15:40:18.589874 28062 raft_consensus.cc:2454] T 00000000000000000000000000000000 P c064bc9a409c478cb810307783d32488 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8ba587d8bdca4e27ac7a45bbf5f8a260 in term 1.
I20260809 15:40:18.590000 28171 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P cbe6b1b7953e4d8ca41ba1fb96553973: No bootstrap required, opened a new log
I20260809 15:40:18.591902 27955 leader_election.cc:302] T 00000000000000000000000000000000 P 8ba587d8bdca4e27ac7a45bbf5f8a260 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8ba587d8bdca4e27ac7a45bbf5f8a260, c064bc9a409c478cb810307783d32488; no voters:
I20260809 15:40:18.592190 28181 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 8ba587d8bdca4e27ac7a45bbf5f8a260 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 15:40:18.592451 28181 raft_consensus.cc:686] T 00000000000000000000000000000000 P 8ba587d8bdca4e27ac7a45bbf5f8a260 [term 1 LEADER]: Becoming Leader. State: Replica: 8ba587d8bdca4e27ac7a45bbf5f8a260, State: Running, Role: LEADER
I20260809 15:40:18.592765 28181 consensus_queue.cc:237] T 00000000000000000000000000000000 P 8ba587d8bdca4e27ac7a45bbf5f8a260 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8ba587d8bdca4e27ac7a45bbf5f8a260" member_type: VOTER last_known_addr { host: "127.3.89.254" port: 32885 } } peers { permanent_uuid: "c064bc9a409c478cb810307783d32488" member_type: VOTER last_known_addr { host: "127.3.89.253" port: 46795 } } peers { permanent_uuid: "cbe6b1b7953e4d8ca41ba1fb96553973" member_type: VOTER last_known_addr { host: "127.3.89.252" port: 39119 } }
I20260809 15:40:18.594544 28171 raft_consensus.cc:348] T 00000000000000000000000000000000 P cbe6b1b7953e4d8ca41ba1fb96553973 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8ba587d8bdca4e27ac7a45bbf5f8a260" member_type: VOTER last_known_addr { host: "127.3.89.254" port: 32885 } } peers { permanent_uuid: "c064bc9a409c478cb810307783d32488" member_type: VOTER last_known_addr { host: "127.3.89.253" port: 46795 } } peers { permanent_uuid: "cbe6b1b7953e4d8ca41ba1fb96553973" member_type: VOTER last_known_addr { host: "127.3.89.252" port: 39119 } }
I20260809 15:40:18.594882 28171 raft_consensus.cc:729] T 00000000000000000000000000000000 P cbe6b1b7953e4d8ca41ba1fb96553973 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: cbe6b1b7953e4d8ca41ba1fb96553973, State: Initialized, Role: FOLLOWER
I20260809 15:40:18.595302 28171 consensus_queue.cc:260] T 00000000000000000000000000000000 P cbe6b1b7953e4d8ca41ba1fb96553973 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8ba587d8bdca4e27ac7a45bbf5f8a260" member_type: VOTER last_known_addr { host: "127.3.89.254" port: 32885 } } peers { permanent_uuid: "c064bc9a409c478cb810307783d32488" member_type: VOTER last_known_addr { host: "127.3.89.253" port: 46795 } } peers { permanent_uuid: "cbe6b1b7953e4d8ca41ba1fb96553973" member_type: VOTER last_known_addr { host: "127.3.89.252" port: 39119 } }
I20260809 15:40:18.596707 28171 sys_catalog.cc:564] T 00000000000000000000000000000000 P cbe6b1b7953e4d8ca41ba1fb96553973 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 15:40:18.596786 28142 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "8ba587d8bdca4e27ac7a45bbf5f8a260" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: true dest_uuid: "cbe6b1b7953e4d8ca41ba1fb96553973"
I20260809 15:40:18.596978 28142 raft_consensus.cc:3037] T 00000000000000000000000000000000 P cbe6b1b7953e4d8ca41ba1fb96553973 [term 0 FOLLOWER]: Advancing to term 1
I20260809 15:40:18.598568 28214 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8ba587d8bdca4e27ac7a45bbf5f8a260 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 8ba587d8bdca4e27ac7a45bbf5f8a260. Latest consensus state: current_term: 1 leader_uuid: "8ba587d8bdca4e27ac7a45bbf5f8a260" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8ba587d8bdca4e27ac7a45bbf5f8a260" member_type: VOTER last_known_addr { host: "127.3.89.254" port: 32885 } } peers { permanent_uuid: "c064bc9a409c478cb810307783d32488" member_type: VOTER last_known_addr { host: "127.3.89.253" port: 46795 } } peers { permanent_uuid: "cbe6b1b7953e4d8ca41ba1fb96553973" member_type: VOTER last_known_addr { host: "127.3.89.252" port: 39119 } } }
I20260809 15:40:18.598642 28142 raft_consensus.cc:2454] T 00000000000000000000000000000000 P cbe6b1b7953e4d8ca41ba1fb96553973 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8ba587d8bdca4e27ac7a45bbf5f8a260 in term 1.
I20260809 15:40:18.598778 28214 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8ba587d8bdca4e27ac7a45bbf5f8a260 [sys.catalog]: This master's current role is: LEADER
I20260809 15:40:18.605970 28228 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 15:40:18.607118 28216 sys_catalog.cc:455] T 00000000000000000000000000000000 P cbe6b1b7953e4d8ca41ba1fb96553973 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8ba587d8bdca4e27ac7a45bbf5f8a260" member_type: VOTER last_known_addr { host: "127.3.89.254" port: 32885 } } peers { permanent_uuid: "c064bc9a409c478cb810307783d32488" member_type: VOTER last_known_addr { host: "127.3.89.253" port: 46795 } } peers { permanent_uuid: "cbe6b1b7953e4d8ca41ba1fb96553973" member_type: VOTER last_known_addr { host: "127.3.89.252" port: 39119 } } }
I20260809 15:40:18.607264 28216 sys_catalog.cc:458] T 00000000000000000000000000000000 P cbe6b1b7953e4d8ca41ba1fb96553973 [sys.catalog]: This master's current role is: FOLLOWER
I20260809 15:40:18.608115 28228 catalog_manager.cc:1484] Initializing Kudu cluster ID...
W20260809 15:40:18.610253 28227 catalog_manager.cc:1558] T 00000000000000000000000000000000 P cbe6b1b7953e4d8ca41ba1fb96553973: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
I20260809 15:40:18.610409 28012 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:40:18.599951 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36168 (local address 127.3.89.254:32885)
0809 15:40:18.600052 (+ 101us) server_negotiation.cc:206] Beginning negotiation
0809 15:40:18.600056 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 15:40:18.600909 (+ 853us) server_negotiation.cc:407] Connection header received
0809 15:40:18.601099 (+ 190us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 15:40:18.601103 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 15:40:18.601150 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 15:40:18.601201 (+ 51us) server_negotiation.cc:226] Negotiated authn=SASL
0809 15:40:18.602302 (+ 1101us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:40:18.602927 (+ 625us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:40:18.604431 (+ 1504us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:40:18.604722 (+ 291us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:40:18.604783 (+ 61us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:40:18.606774 (+ 1991us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 15:40:18.606778 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 15:40:18.606781 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 15:40:18.606808 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0809 15:40:18.608543 (+ 1735us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 15:40:18.609023 (+ 480us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 15:40:18.609027 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 15:40:18.609028 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 15:40:18.609116 (+ 88us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 15:40:18.609427 (+ 311us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 15:40:18.609431 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 15:40:18.609433 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 15:40:18.609614 (+ 181us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 15:40:18.609755 (+ 141us) server_negotiation.cc:1035] Waiting for connection context
0809 15:40:18.610073 (+ 318us) server_negotiation.cc:299] Negotiation successful
0809 15:40:18.610229 (+ 156us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":36}
I20260809 15:40:18.614382 28096 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:40:18.600370 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35762 (local address 127.3.89.253:46795)
0809 15:40:18.600481 (+ 111us) server_negotiation.cc:206] Beginning negotiation
0809 15:40:18.600485 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 15:40:18.605294 (+ 4809us) server_negotiation.cc:407] Connection header received
0809 15:40:18.605485 (+ 191us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 15:40:18.605490 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 15:40:18.605547 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 15:40:18.605629 (+ 82us) server_negotiation.cc:226] Negotiated authn=SASL
0809 15:40:18.606543 (+ 914us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:40:18.607379 (+ 836us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:40:18.608679 (+ 1300us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:40:18.609045 (+ 366us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:40:18.609117 (+ 72us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:40:18.611261 (+ 2144us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 15:40:18.611264 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 15:40:18.611267 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 15:40:18.611293 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0809 15:40:18.612813 (+ 1520us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 15:40:18.613220 (+ 407us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 15:40:18.613223 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 15:40:18.613225 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 15:40:18.613302 (+ 77us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 15:40:18.613567 (+ 265us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 15:40:18.613570 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 15:40:18.613572 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 15:40:18.613741 (+ 169us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 15:40:18.613867 (+ 126us) server_negotiation.cc:1035] Waiting for connection context
0809 15:40:18.614089 (+ 222us) server_negotiation.cc:299] Negotiation successful
0809 15:40:18.614207 (+ 118us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":45}
W20260809 15:40:18.615028 28227 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260809 15:40:18.620028 28176 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:40:18.599453 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34778 (local address 127.3.89.252:39119)
0809 15:40:18.599595 (+ 142us) server_negotiation.cc:206] Beginning negotiation
0809 15:40:18.599601 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 15:40:18.605621 (+ 6020us) server_negotiation.cc:407] Connection header received
0809 15:40:18.606134 (+ 513us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 15:40:18.606137 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 15:40:18.606184 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 15:40:18.606232 (+ 48us) server_negotiation.cc:226] Negotiated authn=SASL
0809 15:40:18.608368 (+ 2136us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:40:18.609080 (+ 712us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:40:18.615089 (+ 6009us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:40:18.615440 (+ 351us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:40:18.615527 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:40:18.616972 (+ 1445us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 15:40:18.616975 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 15:40:18.616978 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 15:40:18.617010 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0809 15:40:18.618429 (+ 1419us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 15:40:18.618835 (+ 406us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 15:40:18.618838 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 15:40:18.618840 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 15:40:18.618917 (+ 77us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 15:40:18.619232 (+ 315us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 15:40:18.619235 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 15:40:18.619237 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 15:40:18.619410 (+ 173us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 15:40:18.619556 (+ 146us) server_negotiation.cc:1035] Waiting for connection context
0809 15:40:18.619771 (+ 215us) server_negotiation.cc:299] Negotiation successful
0809 15:40:18.619883 (+ 112us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":51}
I20260809 15:40:18.626761 28142 raft_consensus.cc:1261] T 00000000000000000000000000000000 P cbe6b1b7953e4d8ca41ba1fb96553973 [term 1 FOLLOWER]: Refusing update from remote peer 8ba587d8bdca4e27ac7a45bbf5f8a260: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 15:40:18.627454 28181 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 8ba587d8bdca4e27ac7a45bbf5f8a260 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cbe6b1b7953e4d8ca41ba1fb96553973" member_type: VOTER last_known_addr { host: "127.3.89.252" port: 39119 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 15:40:18.627915 28062 raft_consensus.cc:1261] T 00000000000000000000000000000000 P c064bc9a409c478cb810307783d32488 [term 1 FOLLOWER]: Refusing update from remote peer 8ba587d8bdca4e27ac7a45bbf5f8a260: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 15:40:18.631412 28216 sys_catalog.cc:455] T 00000000000000000000000000000000 P cbe6b1b7953e4d8ca41ba1fb96553973 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 8ba587d8bdca4e27ac7a45bbf5f8a260. Latest consensus state: current_term: 1 leader_uuid: "8ba587d8bdca4e27ac7a45bbf5f8a260" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8ba587d8bdca4e27ac7a45bbf5f8a260" member_type: VOTER last_known_addr { host: "127.3.89.254" port: 32885 } } peers { permanent_uuid: "c064bc9a409c478cb810307783d32488" member_type: VOTER last_known_addr { host: "127.3.89.253" port: 46795 } } peers { permanent_uuid: "cbe6b1b7953e4d8ca41ba1fb96553973" member_type: VOTER last_known_addr { host: "127.3.89.252" port: 39119 } } }
I20260809 15:40:18.631528 28216 sys_catalog.cc:458] T 00000000000000000000000000000000 P cbe6b1b7953e4d8ca41ba1fb96553973 [sys.catalog]: This master's current role is: FOLLOWER
I20260809 15:40:18.631690 28181 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 8ba587d8bdca4e27ac7a45bbf5f8a260 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c064bc9a409c478cb810307783d32488" member_type: VOTER last_known_addr { host: "127.3.89.253" port: 46795 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 15:40:18.636546 28182 sys_catalog.cc:455] T 00000000000000000000000000000000 P c064bc9a409c478cb810307783d32488 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 8ba587d8bdca4e27ac7a45bbf5f8a260. Latest consensus state: current_term: 1 leader_uuid: "8ba587d8bdca4e27ac7a45bbf5f8a260" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8ba587d8bdca4e27ac7a45bbf5f8a260" member_type: VOTER last_known_addr { host: "127.3.89.254" port: 32885 } } peers { permanent_uuid: "c064bc9a409c478cb810307783d32488" member_type: VOTER last_known_addr { host: "127.3.89.253" port: 46795 } } peers { permanent_uuid: "cbe6b1b7953e4d8ca41ba1fb96553973" member_type: VOTER last_known_addr { host: "127.3.89.252" port: 39119 } } }
I20260809 15:40:18.636677 28182 sys_catalog.cc:458] T 00000000000000000000000000000000 P c064bc9a409c478cb810307783d32488 [sys.catalog]: This master's current role is: FOLLOWER
I20260809 15:40:18.639593 28181 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8ba587d8bdca4e27ac7a45bbf5f8a260 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "8ba587d8bdca4e27ac7a45bbf5f8a260" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8ba587d8bdca4e27ac7a45bbf5f8a260" member_type: VOTER last_known_addr { host: "127.3.89.254" port: 32885 } } peers { permanent_uuid: "c064bc9a409c478cb810307783d32488" member_type: VOTER last_known_addr { host: "127.3.89.253" port: 46795 } } peers { permanent_uuid: "cbe6b1b7953e4d8ca41ba1fb96553973" member_type: VOTER last_known_addr { host: "127.3.89.252" port: 39119 } } }
I20260809 15:40:18.639714 28181 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8ba587d8bdca4e27ac7a45bbf5f8a260 [sys.catalog]: This master's current role is: LEADER
I20260809 15:40:18.639830 28233 mvcc.cc:205] Tried to move back new op lower bound from 7316643916286480384 to 7316643916159365120. Current Snapshot: MvccSnapshot[applied={T|T < 7316643916286480384}]
I20260809 15:40:18.640949 28182 sys_catalog.cc:455] T 00000000000000000000000000000000 P c064bc9a409c478cb810307783d32488 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "8ba587d8bdca4e27ac7a45bbf5f8a260" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8ba587d8bdca4e27ac7a45bbf5f8a260" member_type: VOTER last_known_addr { host: "127.3.89.254" port: 32885 } } peers { permanent_uuid: "c064bc9a409c478cb810307783d32488" member_type: VOTER last_known_addr { host: "127.3.89.253" port: 46795 } } peers { permanent_uuid: "cbe6b1b7953e4d8ca41ba1fb96553973" member_type: VOTER last_known_addr { host: "127.3.89.252" port: 39119 } } }
I20260809 15:40:18.641055 28182 sys_catalog.cc:458] T 00000000000000000000000000000000 P c064bc9a409c478cb810307783d32488 [sys.catalog]: This master's current role is: FOLLOWER
I20260809 15:40:18.644845 28237 mvcc.cc:205] Tried to move back new op lower bound from 7316643916286480384 to 7316643916159365120. Current Snapshot: MvccSnapshot[applied={T|T < 7316643916286480384}]
I20260809 15:40:18.647619 28214 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8ba587d8bdca4e27ac7a45bbf5f8a260 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "8ba587d8bdca4e27ac7a45bbf5f8a260" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8ba587d8bdca4e27ac7a45bbf5f8a260" member_type: VOTER last_known_addr { host: "127.3.89.254" port: 32885 } } peers { permanent_uuid: "c064bc9a409c478cb810307783d32488" member_type: VOTER last_known_addr { host: "127.3.89.253" port: 46795 } } peers { permanent_uuid: "cbe6b1b7953e4d8ca41ba1fb96553973" member_type: VOTER last_known_addr { host: "127.3.89.252" port: 39119 } } }
I20260809 15:40:18.647735 28214 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8ba587d8bdca4e27ac7a45bbf5f8a260 [sys.catalog]: This master's current role is: LEADER
I20260809 15:40:18.648741 28216 sys_catalog.cc:455] T 00000000000000000000000000000000 P cbe6b1b7953e4d8ca41ba1fb96553973 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "8ba587d8bdca4e27ac7a45bbf5f8a260" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8ba587d8bdca4e27ac7a45bbf5f8a260" member_type: VOTER last_known_addr { host: "127.3.89.254" port: 32885 } } peers { permanent_uuid: "c064bc9a409c478cb810307783d32488" member_type: VOTER last_known_addr { host: "127.3.89.253" port: 46795 } } peers { permanent_uuid: "cbe6b1b7953e4d8ca41ba1fb96553973" member_type: VOTER last_known_addr { host: "127.3.89.252" port: 39119 } } }
I20260809 15:40:18.648849 28216 sys_catalog.cc:458] T 00000000000000000000000000000000 P cbe6b1b7953e4d8ca41ba1fb96553973 [sys.catalog]: This master's current role is: FOLLOWER
I20260809 15:40:18.650091 28228 catalog_manager.cc:1347] Generated new cluster ID: 16a836f4db73424893f12bc1b9a2a714
I20260809 15:40:18.650153 28228 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 15:40:18.663115 28234 mvcc.cc:205] Tried to move back new op lower bound from 7316643916286480384 to 7316643916159365120. Current Snapshot: MvccSnapshot[applied={T|T < 7316643916286480384}]
I20260809 15:40:18.675443 28228 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 15:40:18.676810 28228 catalog_manager.cc:1504] Loading token signing keys...
I20260809 15:40:18.695214 28228 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 8ba587d8bdca4e27ac7a45bbf5f8a260: Generated new TSK 0
I20260809 15:40:18.698632 28228 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260809 15:40:18.738735 28012 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 15:40:18.735080 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36176 (local address 127.3.89.254:32885)
0809 15:40:18.735215 (+ 135us) server_negotiation.cc:206] Beginning negotiation
0809 15:40:18.735230 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0809 15:40:18.737354 (+ 2124us) server_negotiation.cc:407] Connection header received
0809 15:40:18.737789 (+ 435us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 15:40:18.737792 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 15:40:18.737850 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 15:40:18.737907 (+ 57us) server_negotiation.cc:226] Negotiated authn=SASL
0809 15:40:18.738590 (+ 683us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:36176: BlockingRecv error: recv got EOF from 127.0.0.1:36176 (error 108)
Metrics: {"server-negotiator.queue_time_us":53}
W20260809 15:40:18.740316 28096 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 15:40:18.735888 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35774 (local address 127.3.89.253:46795)
0809 15:40:18.736036 (+ 148us) server_negotiation.cc:206] Beginning negotiation
0809 15:40:18.736040 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 15:40:18.737697 (+ 1657us) server_negotiation.cc:407] Connection header received
0809 15:40:18.739546 (+ 1849us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 15:40:18.739549 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 15:40:18.739600 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 15:40:18.739649 (+ 49us) server_negotiation.cc:226] Negotiated authn=SASL
0809 15:40:18.740180 (+ 531us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:35774: BlockingRecv error: recv got EOF from 127.0.0.1:35774 (error 108)
Metrics: {"server-negotiator.queue_time_us":63}
W20260809 15:40:18.741611 28176 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 15:40:18.736660 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34794 (local address 127.3.89.252:39119)
0809 15:40:18.736790 (+ 130us) server_negotiation.cc:206] Beginning negotiation
0809 15:40:18.736794 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 15:40:18.739366 (+ 2572us) server_negotiation.cc:407] Connection header received
0809 15:40:18.740814 (+ 1448us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 15:40:18.740817 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 15:40:18.740870 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 15:40:18.740923 (+ 53us) server_negotiation.cc:226] Negotiated authn=SASL
0809 15:40:18.741485 (+ 562us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:34794: BlockingRecv error: recv got EOF from 127.0.0.1:34794 (error 108)
Metrics: {"server-negotiator.queue_time_us":47}
W20260809 15:40:18.746832 28096 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 15:40:18.745152 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35780 (local address 127.3.89.253:46795)
0809 15:40:18.745268 (+ 116us) server_negotiation.cc:206] Beginning negotiation
0809 15:40:18.745272 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 15:40:18.745774 (+ 502us) server_negotiation.cc:407] Connection header received
0809 15:40:18.746022 (+ 248us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 15:40:18.746026 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 15:40:18.746105 (+ 79us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 15:40:18.746164 (+ 59us) server_negotiation.cc:226] Negotiated authn=SASL
0809 15:40:18.746728 (+ 564us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:35780: BlockingRecv error: recv got EOF from 127.0.0.1:35780 (error 108)
Metrics: {"server-negotiator.queue_time_us":43}
W20260809 15:40:18.752285 28096 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 15:40:18.750471 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35788 (local address 127.3.89.253:46795)
0809 15:40:18.750591 (+ 120us) server_negotiation.cc:206] Beginning negotiation
0809 15:40:18.750596 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 15:40:18.751195 (+ 599us) server_negotiation.cc:407] Connection header received
0809 15:40:18.751430 (+ 235us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 15:40:18.751434 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 15:40:18.751510 (+ 76us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 15:40:18.751565 (+ 55us) server_negotiation.cc:226] Negotiated authn=SASL
0809 15:40:18.752152 (+ 587us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:35788: BlockingRecv error: recv got EOF from 127.0.0.1:35788 (error 108)
Metrics: {"server-negotiator.queue_time_us":41}
W20260809 15:40:18.757979 28096 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 15:40:18.756398 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35800 (local address 127.3.89.253:46795)
0809 15:40:18.756503 (+ 105us) server_negotiation.cc:206] Beginning negotiation
0809 15:40:18.756507 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 15:40:18.757027 (+ 520us) server_negotiation.cc:407] Connection header received
0809 15:40:18.757241 (+ 214us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 15:40:18.757252 (+ 11us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 15:40:18.757305 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 15:40:18.757361 (+ 56us) server_negotiation.cc:226] Negotiated authn=SASL
0809 15:40:18.757858 (+ 497us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:35800: BlockingRecv error: recv got EOF from 127.0.0.1:35800 (error 108)
Metrics: {"server-negotiator.queue_time_us":47}
W20260809 15:40:18.765488 28096 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 15:40:18.764030 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35804 (local address 127.3.89.253:46795)
0809 15:40:18.764150 (+ 120us) server_negotiation.cc:206] Beginning negotiation
0809 15:40:18.764153 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0809 15:40:18.764646 (+ 493us) server_negotiation.cc:407] Connection header received
0809 15:40:18.764816 (+ 170us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 15:40:18.764819 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 15:40:18.764865 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 15:40:18.764912 (+ 47us) server_negotiation.cc:226] Negotiated authn=SASL
0809 15:40:18.765392 (+ 480us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:35804: BlockingRecv error: recv got EOF from 127.0.0.1:35804 (error 108)
Metrics: {"server-negotiator.queue_time_us":62}
W20260809 15:40:18.777266 28096 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 15:40:18.775748 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35806 (local address 127.3.89.253:46795)
0809 15:40:18.775867 (+ 119us) server_negotiation.cc:206] Beginning negotiation
0809 15:40:18.775871 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 15:40:18.776384 (+ 513us) server_negotiation.cc:407] Connection header received
0809 15:40:18.776560 (+ 176us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 15:40:18.776564 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 15:40:18.776611 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 15:40:18.776661 (+ 50us) server_negotiation.cc:226] Negotiated authn=SASL
0809 15:40:18.777162 (+ 501us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:35806: BlockingRecv error: recv got EOF from 127.0.0.1:35806 (error 108)
Metrics: {"server-negotiator.queue_time_us":43}
W20260809 15:40:18.797801 28096 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 15:40:18.795925 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35814 (local address 127.3.89.253:46795)
0809 15:40:18.796050 (+ 125us) server_negotiation.cc:206] Beginning negotiation
0809 15:40:18.796055 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 15:40:18.796668 (+ 613us) server_negotiation.cc:407] Connection header received
0809 15:40:18.796989 (+ 321us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 15:40:18.796993 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 15:40:18.797061 (+ 68us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 15:40:18.797171 (+ 110us) server_negotiation.cc:226] Negotiated authn=SASL
0809 15:40:18.797665 (+ 494us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:35814: BlockingRecv error: recv got EOF from 127.0.0.1:35814 (error 108)
Metrics: {"server-negotiator.queue_time_us":51}
W20260809 15:40:18.833043 28096 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 15:40:18.831723 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35826 (local address 127.3.89.253:46795)
0809 15:40:18.831899 (+ 176us) server_negotiation.cc:206] Beginning negotiation
0809 15:40:18.831903 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 15:40:18.832079 (+ 176us) server_negotiation.cc:407] Connection header received
0809 15:40:18.832246 (+ 167us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 15:40:18.832249 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 15:40:18.832296 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 15:40:18.832372 (+ 76us) server_negotiation.cc:226] Negotiated authn=SASL
0809 15:40:18.832895 (+ 523us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:35826: BlockingRecv error: recv got EOF from 127.0.0.1:35826 (error 108)
Metrics: {"server-negotiator.queue_time_us":63}
W20260809 15:40:18.900631 28096 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 15:40:18.899341 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35828 (local address 127.3.89.253:46795)
0809 15:40:18.899487 (+ 146us) server_negotiation.cc:206] Beginning negotiation
0809 15:40:18.899491 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 15:40:18.899699 (+ 208us) server_negotiation.cc:407] Connection header received
0809 15:40:18.899804 (+ 105us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 15:40:18.899807 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 15:40:18.899858 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 15:40:18.899940 (+ 82us) server_negotiation.cc:226] Negotiated authn=SASL
0809 15:40:18.900499 (+ 559us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:35828: BlockingRecv error: recv got EOF from 127.0.0.1:35828 (error 108)
Metrics: {"server-negotiator.queue_time_us":59}
W20260809 15:40:19.032655 28096 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 15:40:19.031401 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35842 (local address 127.3.89.253:46795)
0809 15:40:19.031527 (+ 126us) server_negotiation.cc:206] Beginning negotiation
0809 15:40:19.031532 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 15:40:19.031763 (+ 231us) server_negotiation.cc:407] Connection header received
0809 15:40:19.031917 (+ 154us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 15:40:19.031922 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 15:40:19.031979 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 15:40:19.032075 (+ 96us) server_negotiation.cc:226] Negotiated authn=SASL
0809 15:40:19.032543 (+ 468us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:35842: BlockingRecv error: recv got EOF from 127.0.0.1:35842 (error 108)
Metrics: {"server-negotiator.queue_time_us":32}
W20260809 15:40:19.292281 28096 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 15:40:19.291111 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35858 (local address 127.3.89.253:46795)
0809 15:40:19.291247 (+ 136us) server_negotiation.cc:206] Beginning negotiation
0809 15:40:19.291251 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 15:40:19.291382 (+ 131us) server_negotiation.cc:407] Connection header received
0809 15:40:19.291529 (+ 147us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 15:40:19.291531 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 15:40:19.291580 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 15:40:19.291661 (+ 81us) server_negotiation.cc:226] Negotiated authn=SASL
0809 15:40:19.292159 (+ 498us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:35858: BlockingRecv error: recv got EOF from 127.0.0.1:35858 (error 108)
Metrics: {"server-negotiator.queue_time_us":57}
I20260809 15:40:19.574615 28200 catalog_manager.cc:1259] Loaded cluster ID: 16a836f4db73424893f12bc1b9a2a714
I20260809 15:40:19.574702 28200 catalog_manager.cc:1552] T 00000000000000000000000000000000 P c064bc9a409c478cb810307783d32488: loading cluster ID for follower catalog manager: success
I20260809 15:40:19.576407 28200 catalog_manager.cc:1574] T 00000000000000000000000000000000 P c064bc9a409c478cb810307783d32488: acquiring CA information for follower catalog manager: success
I20260809 15:40:19.577562 28200 catalog_manager.cc:1602] T 00000000000000000000000000000000 P c064bc9a409c478cb810307783d32488: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260809 15:40:19.616401 28227 catalog_manager.cc:1259] Loaded cluster ID: 16a836f4db73424893f12bc1b9a2a714
I20260809 15:40:19.616487 28227 catalog_manager.cc:1552] T 00000000000000000000000000000000 P cbe6b1b7953e4d8ca41ba1fb96553973: loading cluster ID for follower catalog manager: success
I20260809 15:40:19.618115 28227 catalog_manager.cc:1574] T 00000000000000000000000000000000 P cbe6b1b7953e4d8ca41ba1fb96553973: acquiring CA information for follower catalog manager: success
I20260809 15:40:19.619263 28227 catalog_manager.cc:1602] T 00000000000000000000000000000000 P cbe6b1b7953e4d8ca41ba1fb96553973: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260809 15:40:19.811152 28310 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:40:19.807085 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35874 (local address 127.3.89.253:46795)
0809 15:40:19.807334 (+ 249us) server_negotiation.cc:206] Beginning negotiation
0809 15:40:19.807340 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 15:40:19.807412 (+ 72us) server_negotiation.cc:407] Connection header received
0809 15:40:19.807581 (+ 169us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 15:40:19.807584 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 15:40:19.807727 (+ 143us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 15:40:19.807847 (+ 120us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 15:40:19.808125 (+ 278us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:40:19.808682 (+ 557us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:40:19.809569 (+ 887us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:40:19.809799 (+ 230us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:40:19.809910 (+ 111us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:40:19.810235 (+ 325us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 15:40:19.810677 (+ 442us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 15:40:19.810806 (+ 129us) server_negotiation.cc:1035] Waiting for connection context
0809 15:40:19.810867 (+ 61us) server_negotiation.cc:299] Negotiation successful
0809 15:40:19.810923 (+ 56us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":165,"thread_start_us":66,"threads_started":1}
I20260809 15:40:19.817566 28316 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:40:19.814083 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34808 (local address 127.3.89.252:39119)
0809 15:40:19.814327 (+ 244us) server_negotiation.cc:206] Beginning negotiation
0809 15:40:19.814331 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 15:40:19.814344 (+ 13us) server_negotiation.cc:407] Connection header received
0809 15:40:19.814418 (+ 74us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 15:40:19.814420 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 15:40:19.814519 (+ 99us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 15:40:19.814614 (+ 95us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 15:40:19.814831 (+ 217us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:40:19.815398 (+ 567us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:40:19.816173 (+ 775us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:40:19.816392 (+ 219us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:40:19.816465 (+ 73us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:40:19.816740 (+ 275us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 15:40:19.817110 (+ 370us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 15:40:19.817196 (+ 86us) server_negotiation.cc:1035] Waiting for connection context
0809 15:40:19.817293 (+ 97us) server_negotiation.cc:299] Negotiation successful
0809 15:40:19.817361 (+ 68us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":163,"thread_start_us":67,"threads_started":1}
I20260809 15:40:19.824414 28310 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:40:19.820691 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35886 (local address 127.3.89.253:46795)
0809 15:40:19.820804 (+ 113us) server_negotiation.cc:206] Beginning negotiation
0809 15:40:19.820808 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 15:40:19.821210 (+ 402us) server_negotiation.cc:407] Connection header received
0809 15:40:19.821407 (+ 197us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 15:40:19.821412 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 15:40:19.821462 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 15:40:19.821543 (+ 81us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 15:40:19.821912 (+ 369us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:40:19.822383 (+ 471us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:40:19.823302 (+ 919us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:40:19.823519 (+ 217us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:40:19.823629 (+ 110us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:40:19.823960 (+ 331us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 15:40:19.824028 (+ 68us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 15:40:19.824084 (+ 56us) server_negotiation.cc:1035] Waiting for connection context
0809 15:40:19.824202 (+ 118us) server_negotiation.cc:299] Negotiation successful
0809 15:40:19.824284 (+ 82us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":35}
I20260809 15:40:19.824997 28316 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:40:19.820564 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34824 (local address 127.3.89.252:39119)
0809 15:40:19.820679 (+ 115us) server_negotiation.cc:206] Beginning negotiation
0809 15:40:19.820683 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 15:40:19.821117 (+ 434us) server_negotiation.cc:407] Connection header received
0809 15:40:19.821203 (+ 86us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 15:40:19.821206 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 15:40:19.821256 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 15:40:19.821326 (+ 70us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 15:40:19.821696 (+ 370us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:40:19.822424 (+ 728us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:40:19.823889 (+ 1465us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:40:19.824190 (+ 301us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:40:19.824289 (+ 99us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:40:19.824532 (+ 243us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 15:40:19.824602 (+ 70us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 15:40:19.824667 (+ 65us) server_negotiation.cc:1035] Waiting for connection context
0809 15:40:19.824803 (+ 136us) server_negotiation.cc:299] Negotiation successful
0809 15:40:19.824861 (+ 58us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":40}
I20260809 15:40:19.826269 28325 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 15:40:19.820451 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36180 (local address 127.3.89.254:32885)
0809 15:40:19.820822 (+ 371us) server_negotiation.cc:206] Beginning negotiation
0809 15:40:19.820828 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 15:40:19.820843 (+ 15us) server_negotiation.cc:407] Connection header received
0809 15:40:19.821012 (+ 169us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 15:40:19.821017 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 15:40:19.821182 (+ 165us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 15:40:19.821324 (+ 142us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 15:40:19.821552 (+ 228us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:40:19.822408 (+ 856us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:40:19.824370 (+ 1962us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 15:40:19.824720 (+ 350us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 15:40:19.824815 (+ 95us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 15:40:19.825573 (+ 758us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 15:40:19.825869 (+ 296us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 15:40:19.825978 (+ 109us) server_negotiation.cc:1035] Waiting for connection context
0809 15:40:19.826057 (+ 79us) server_negotiation.cc:299] Negotiation successful
0809 15:40:19.826114 (+ 57us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":293,"thread_start_us":94,"threads_started":1}
I20260809 15:40:19.829217 3431 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskm314YR/build/debug/bin/kudu with pid 27934
I20260809 15:40:19.837761 3431 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskm314YR/build/debug/bin/kudu with pid 28018
I20260809 15:40:19.845944 3431 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskm314YR/build/debug/bin/kudu with pid 28099
2026-08-09T15:40:19Z chronyd exiting
[ OK ] ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys/1 (3166 ms)
[----------] 2 tests from ConnectToFollowerMasterTest (6298 ms total)
[----------] Global test environment tear-down
[==========] 56 tests from 4 test suites ran. (286997 ms total)
[ PASSED ] 55 tests.
[ FAILED ] 1 test, listed below:
[ FAILED ] SecurityITest.TestJwtMiniCluster
1 FAILED TEST